Digital image acquisition and processing system

ABSTRACT

A digital image acquisition and processing system that automatically corrects dust artifact regions within acquired images by compiling a statistical dust map from multiple images acquired under different image acquisition conditions is provided. The system includes a digital camera and an external processing device. The camera includes an optical system for acquiring an image including a lens assembly and an aperture stop, and an electronic sensor array disposed approximately at an image focal plane of the optical system for collecting image data according to spectral information associated with multiple pixels that collectively correspond to the image. The external processing device couples with the digital camera to receive the image data or converted image data internally processed within the digital camera. The external device includes digital processing electronics including a processor for processing the image data according to programming instructions. The system includes one or more memories having programming instructions stored therein for performing a method of automatic image correction of dust defect regions.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is related to a family of patent applications filed onthe same day by the same inventors including those entitled: (1)Automated Statistical Self-Calibrating Detection and Removal ofBlemishes in Digital Images Based On Multiple Occurrences Of Dust InImages; (2) Automated Statistical Self-Calibrating Detection and Removalof Blemishes in Digital Images Based on a Dust Map Developed From ActualImage Data; (3) Automated Statistical Self-Calibrating Detection andRemoval of Blemishes in Digital Images Dependent Upon Changes inExtracted Parameter Values; (4) Automated Statistical Self-CalibratingDetection and Removal of Blemishes in Digital Images Based onDetermining Probabilities Based On Image Analysis Of Single Images; (5)Determination Of Need To Service A Camera Based On Detection ofBlemishes in Digital Images; (6) Method Of Detecting and Correcting Dustin Digital Images Based On Aura And Shadow Region Analysis; (7) DigitalCamera; and (8) Digital Image Acquisition And Processing System, whichare all hereby incorporated by reference.

BACKGROUND

1. Field of the Invention

This invention related to digital photography and in particular,automated means of removing blemish artifacts from images captured anddigitized on a digital process.

2. Description of the Related Art

Many problems are caused by dust in particular and blemishes in generalon imaging devices in general and digital imaging devices in particular.In the past, two distinct forms of image processing included providingmeans to detect and locate dust, scratches or similar defects andproviding means to remedy the distortions caused by the defects to animage. It is desired to have an advantageous system that combines thesefunctions, and can automatically detect and correct for the effects ofdust, scratches and other optical blemishes.

Dust has been a problem in scanning devices for a long time. Variousaspects of the scanning process, enhanced by image processing techniqueswhere appropriate, have been employed to provide means for the detectionof dust or defects relating to document or image/film scanners. Thesedevices form an image by moving a 1D sensor pixel array across adocument platen, or in some cases, the document platten, withdocument/image is moved in under the sensor array. The physics andimplementation of these devices differ significantly from those of afield-based sensor or camera-style device. It is desired particularly tohave dust and/or blemish detection and correction techniques forfield-based or camera-style acquisition devices.

Image correction has been studied in relation to display devices, outputapparatuses such as printers, and digital sensors. Image correction ofdust artifacts can be used to recreate missing data, also referred to asin-painting or restoration, or undoing degradation of data, which stillremains in the image, also referred to as image enhancement. It isdesired to have a system including a digital camera and an externaldevice or apparatus that can facilitate a defect detection and/orcorrection process involving sophisticated and automated computerizedprogramming techniques.

SUMMARY OF THE INVENTION

A digital image acquisition and processing system that automaticallycorrects dust artifact regions within acquired images by compiling astatistical dust map from multiple images acquired under different imageacquisition conditions is provided. The system includes a digital cameraand an external processing device. The camera includes an optical systemfor acquiring an image including a lens assembly and an aperture stop,and an electronic sensor array disposed approximately at an image focalplane of the optical system for collecting image data according tospectral information associated with multiple pixels that collectivelycorrespond to the image. The external processing device couples with thedigital camera to receive the image data or converted image datainternally processed within the digital camera. The external deviceincludes digital processing electronics including a processor forprocessing the image data according to programming instructions. Thesystem includes one or more memories having programming instructionsstored therein for performing a method of automatic image correction ofdust defect regions.

The image correction method may include acquiring multiple originaldigital images with the digital acquisition device. Probabilities aredetermined that certain pixels correspond to dust artifact regionswithin the images based at least in part on a comparison of suspecteddust artifact regions within two or more of the images. Probable dustartifact regions are associated with one or more values of one or moreextracted parameters relating to the lens assembly of the digitalacquisition device when the images were acquired. A statistical dust mapis formed including mapped dust regions based on the dust artifactdetermining and associating. Pixels corresponding to correlated dustartifact regions are corrected within further digitally-acquired imagesbased on the associated statistical dust map.

The determining of probabilities may further include statisticallycombining multiple individual probabilities based on each of the regionswithin two or more of the images. The method may include eliminatingcertain suspected dust artifact regions as having a probability below afirst threshold value. The method may further include judging certainprobable dust artifact regions as having a probability above a thresholdvalue such as to be subject to said correcting operation. Theprobability determining may include weighting suspected dust artifactregions according to one or more predetermined probability weightingassessment conditions.

The further digitally-acquired images may include different images thanthe originally acquired images. The method may further includecorrecting pixels corresponding to correlated dust artifact regionswithin each of the original images based on the associated statisticaldust map. The method may include correcting pixels corresponding tocorrelated dust artifact regions within the original images based on theassociated statistical dust map. The method may includedigitally-acquiring additional images with the digital camera, repeatingthe determining and associating, and updating the statistical dust mapincluding updating the mapped dust regions based on the additional dustartifact determining and associating.

The determining may includes dynamically updating the probabilitiesbased on comparisons with suspected equivalent dust artifact regionswithin the further digitally-acquired images. The determining of theprobabilities may be further based on a pixel analysis of the suspecteddust artifact regions in view of predetermined characteristicsindicative of the presence of a dust artifact region. The method mayinclude validating whether the further digitally-acquired image hasnon-contradicting data that the probability that certain pixelscorrespond to dust artifact regions within the furtherdigitally-acquired image prior to correcting pixels corresponding tocorrelated dust artifact regions within further digitally-acquiredimages based on the associated statistical dust map.

The image correction method may be performed on a processed image afterbeing converted from raw format to a known red, green, bluerepresentation. The correcting may include replacing pixels within theone or more digitally-acquired images with new pixels. The correctingmay include enhancing the values of pixels within the one or moredigitally-acquired images.

The dust artifact determining may include loading the statistical dustmap, loading extracted parameter information of a present image,performing calculations within the statistical dust map having extractedparameter variable-dependencies, and comparing dust artifact detectiondata with the extracted parameter dependent statistical dust map data.The dust artifact determining may also include loading the statisticaldust map, loading extracted parameter information of a present image,performing a calculation for relating the statistical dust map with thepresent image according to a selected value of an extracted parameterwhich is otherwise uncorrelated between the present image and the dustmap, and comparing dust artifact detection data with the now correlatedstatistical dust map data. The suspected dust artifact regions of atleast two images may include shadow regions and aura regions, andwherein the method may include a first comparison of the shadow regionsand a second comparison of the aura regions.

The image correction method according to a further aspect includesdigitally-acquiring one or more original images with the digital camera.Probabilities are determined that certain pixels correspond to dustartifact regions within the one or more digitally-acquired images. Thedust artifact regions are associated with one or more extractedparameters relating to the optical system when the one or more imageswere acquired. A statistical dust map is formed including mapped dustregions based on the dust artifact probability determining andassociating. Pixels are corrected corresponding to dust artifact regionswithin each of the one or more original images based on the associatedstatistical dust map.

The method may include digitally-acquiring further images with thedigital camera, repeating the determining and associating, and updatingthe statistical dust map including updating the mapped dust regionsbased on the further dust artifact determining and associating. Thedetermining may include determining probabilities that certain pixelscorrespond to dust artifact regions within the acquired images based atleast in part on a comparison of suspected dust artifact regions withintwo or more digitally-acquired images, or on a pixel analysis of thesuspected dust artifact regions in view of predetermined characteristicsindicative of the presence of a dust artifact region, or both. Thedetermining may be based at least in part on a comparison of suspecteddust artifact regions within two or more digitally-acquired images.

The determining of probabilities may be further based on a pixelanalysis of the suspected dust artifact regions in view of predeterminedcharacteristics indicative of the presence of a dust artifact region.The suspected dust artifact regions of the at least two images mayinclude shadow regions and partially-occluded aura regions, and themethod may include a first correcting of the aura regions and a secondcorrecting of the shadow regions. The dust map may include dust artifactregions with probabilities above a threshold probability while notincluding regions with lower probabilities. The dust map may includedust artifact regions with probabilities above a first threshold value,while not including regions with probabilities below a second thresholdvalue, and where further regions may have a probability between thefirst and second threshold values exist within the dust map, thenfurther image information is combined into the probability analysisbefore determining whether the regions are included as dust artifactregions.

The correcting may further include replacing the pixels within the oneor more digitally-acquired images with new pixels. The correcting mayinclude enhancing the values of pixels within the one or moredigitally-acquired images.

The dust artifact determining operation may include loading thestatistical dust map, loading extracted parameter information of apresent image, performing calculations within the statistical dust maphaving extracted parameter variable-dependencies, and comparing dustartifact detection data with the extracted parameter dependentstatistical dust map data. The dust artifact determining operation mayalso include loading the statistical dust map, loading extractedparameter information of a present image, performing a calculation forrelating the statistical dust map with the present image according to aselected value of an extracted parameter which is otherwise uncorrelatedbetween the present image and the dust map, and comparing dust artifactdetection data with the now correlated statistical dust map data.

A method according to a further aspect includes determining that certainpixels correspond to dust artifact regions within a firstdigitally-acquired image. The dust artifact regions are associated withone or more extracted parameters relating to a variable optical systemof said digital camera when the first image was acquired. An associatedstatistical dust map is formed including mapped dust regions based onthe dust artifact determining and associating operations and knowncorrespondence between effects on dust artifact data of one or moredifferent values of said one or more extracted parameters. Pixels arecorrected corresponding to correlated dust artifact regions within asecond image acquired with said digital camera according to one or moredifferent values of said one or more extracted parameters relating tothe variable optical system based on said associated statistical dustmap.

The first and second images may be acquired of different objects. Pixelsmay be corrected corresponding to correlated dust artifact regionswithin the first image based on the associated statistical dust map. Themethod may include digitally-acquiring additional images with thedigital camera, repeating the determining and associating, and updatingthe statistical dust map including updating the mapped dust regionsbased on the additional dust artifact determining and associating. Thedetermining may include determining probabilities that certain pixelscorrespond to dust artifact regions within the first acquired image. Thedetermining may be based at least in part on a comparison of suspecteddust artifact regions within multiple images including the first imageand at least one other digitally-acquired image.

A method according to a further aspect includes determiningprobabilities that certain pixels within multiple digitally-acquiredimages correspond to one or more dust artifact regions. Probable dustartifact regions are associated with values of one or more extractedparameters relating to a lens system of the digital image acquisitiondevice when the images were acquired. A statistical dust map is formedincluding mapped dust regions based on the dust artifact determining andassociating operations. Digital data are corrected corresponding topixels within one or more digitally-acquired images correlated withmapped dust regions of the statistical dust map.

The method may include repeating the dust artifact determining andassociating operations for further acquired images, while values of theone or more extracted parameters relating to the optical system maydiffer between at least two acquired images. At least two acquiredimages may have been acquired of different objects. The forming of thestatistical dust map may be based upon multiple acquired images and mayinclude mathematically correlating dust artifact regions of the multipleacquired images according to their extracted parameter values includingthe one or more different extracted parameter values of at least twoacquired images.

The method according to a further aspect includes determiningprobabilities of dust artifact regions corresponding to pixels within adigitally-acquired image based at least in part on a pixel analysis ofthe region in view of predetermined characteristics indicative of thepresence of a dust artifact region. The dust artifact regions areassociated with one or more extracted parameters relating to the opticalsystem when the image was acquired. A statistical dust map is formedincluding mapped-dust-regions based on the dust artifact determining andassociating operations. The digital data is corrected corresponding tothe correlated dust artifact regions within the acquired image based onthe associated statistical dust map.

The method may include eliminating certain suspected dust artifactregions as having a probability below a first threshold value. Themethod may further include judging certain probable dust artifactregions as having a probability above a threshold value such as to besubject to the correcting operation. The probability determining mayinclude weighting suspected dust artifact regions according to one ormore predetermined probability weighting assessment conditions. Furtherdigitally-acquired images may include different images than theoriginally acquired images. Pixels corresponding to correlated dustartifact regions may be corrected within each of the original imagesbased on the associated statistical dust map. Pixels corresponding tocorrelated dust artifact regions may be corrected within the originalimages based on the associated statistical dust map.

The determining may include dynamically updating the probabilities basedon comparisons with suspected equivalent dust artifact regions withinthe further digitally-acquired images. The forming may includedynamically updating the statistical dust map based on further pixelanalyses of suspected dust artifact regions in further images in view ofpredetermined characteristics indicative of the presence of a dustartifact region. The suspected dust artifact regions of the at least twoimages may include shadow regions and aura regions, and the method mayinclude a first comparison of the shadow regions and a second comparisonof the aura regions.

The determining of probabilities may include statistically combiningmultiple individual probabilities based on regions within two or moreimages. The method may include validating whether a furtherdigitally-acquired image has non-contradicting data that the probabilitythat certain pixels correspond to dust artifact regions within thefurther digitally-acquired image prior to correcting pixelscorresponding to correlated dust artifact regions within furtherdigitally-acquired images based on the associated statistical dust map.The correcting may include replacing pixels within the one or moredigitally-acquired images with new pixels. The correcting may includeenhancing the values of pixels within the one or more digitally-acquiredimages.

The dust artifact determining operation may include loading thestatistical dust map, loading extracted parameter information of apresent image, performing calculations within the statistical dust maphaving extracted parameter variable-dependencies, and comparing dustartifact detection data with the extracted parameter dependentstatistical dust map data. The dust artifact determining operation mayalso include loading the statistical dust map, loading extractedparameter information of a present image, performing a calculation forrelating the statistical dust map with the present image according to aselected value of an extracted parameter which is otherwise uncorrelatedbetween the present image and the dust map, and comparing dust artifactdetection data with the now correlated statistical dust map data.

The dust artifact regions may include an aura region partially obscuredby dust and a shadow region at least substantially obscured by dustinside the aura region.

A method in accordance with a further aspect includes analyzing pixelswithin one or more acquired digital images according to probabilitydeterminations that such pixels correspond to blemish artifacts. Whethera threshold distribution of blemish artifacts is present is determinedwithin one or more of the digital images. A need for service isindicated when at least the threshold distribution is determined to bepresent.

The method may further include determining probabilities of dustartifact regions corresponding to the pixels within thedigitally-acquired image. The dust artifact regions may be associatedwith one or more extracted parameters relating to the lens assembly whenthe image was acquired. A statistical record may be formed include dustartifact regions based on the dust artifact determining and associating.The threshold distribution may be determined based on predeterminedcharacteristics of the statistical record.

A method according to a further aspect includes acquiring multipleoriginal digital images with the digital acquisition device.Probabilities are determined that certain pixels correspond to dustartifact regions within the images based at least in part on acomparison of suspected dust artifact regions within two or more of theimages. A statistical dust record is formed including probabilities ofdust artifact regions based on the dust artifact determining andassociating. A need for service is determined when the statistical dustrecord indicates that a predetermined threshold dust artifactdistribution is present within digital images acquired with the digitalacquisition device.

The probability determining may include weighting suspected dustartifact regions according to one or more predetermined probabilityweighting assessment conditions. The method may further includeassociating probable dust artifact regions with one or more values ofone or more extracted parameters relating to the lens assembly of thedigital acquisition device when the images were acquired. Thestatistical dust record may be formed from multiple images including atleast two images having different values of one or more extractedparameters that are mathematically correlated based on known effects ofthe different values on dust artifact regions appearing within thedigital images.

A method in accordance with a further aspect includes determiningprobabilities that certain pixels correspond to dust artifact regionswithin a digitally-acquired image based at least in part on a pixelanalysis of the region in view of predetermined characteristicsindicative of the presence of a dust artifact region. A statistical dustrecord is formed including probabilities of dust artifact regions basedon the dust artifact determining and associating operations. A need forservice is determined when the statistical dust record indicates that apredetermined threshold dust artifact distribution is present withindigital images acquired with the digital acquisition device.

The probability determining may include weighting suspected dustartifact regions according to one or more predetermined probabilityweighting assessment conditions. The method may include associatingprobable dust artifact regions with one or more values of one or moreextracted parameters relating to the lens assembly of the digitalacquisition device when the images were acquired. The statistical dustrecord may be formed from multiple images including at least two imageshaving different values of one or more extracted parameters that aremathematically correlated based on known effects of the different valueson dust artifact regions appearing within the digital images.

A method according to another aspect includes determining probabilitiesthat certain pixels within multiple digitally-acquired images correspondto one or more dust artifact regions. A statistical dust map is formedincluding mapped dust artifact regions based on the dust artifactdetermining. The mapped dust artifact regions are separated into auraregions partially obscured by dust and shadow regions substantiallyobscured by dust inside the aura regions. Digital data are correctedcorresponding to pixels within one or more digitally-acquired imagesincluding dust artifact regions separated into aura regions and shadowregions correlated with mapped aura and shadow regions of thestatistical dust map.

Dust artifact region probability determining may include separatingsuspected dust artifact regions into aura regions and shadow regions,and determining separate probabilities whether the aura regions and theshadow regions of the suspected dust artifact regions in fact correspondto dust artifact regions within the images. The correcting operation mayinclude calculating the aura region and the shadow region. The dustartifact region probability determining is based at least in part on acomparison of suspected dust artifact regions within two or more images.The dust artifact region probability determining is based at least inpart on a comparison of suspected dust artifact regions within imageswith predetermined characteristics indicative of the presence of a dustartifact region.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a main workflow of a dust removal process inaccordance with a preferred embodiment.

FIG. 2 a illustrates the creation of a dust map.

FIG. 2 b illustrates an alternative embodiment of the creation of abinary dust map.

FIG. 3 outlines a correlation of a dust map to image shootingparameters.

FIG. 4 a illustrates a procedure for detecting and removing dust fromimages in accordance with a preferred embodiment.

FIG. 4 b illustrates reconstruction in-painting of dust regions.

FIG. 4 c illustrates the occurrence of dust in a picture with high edgeinformation.

FIG. 4 d illustrates the numerical gradient in an image.

FIGS. 4 e-4 f illustrate a spiral procedure of in-painting from theperiphery inwards.

FIG. 5 illustrates a quality control method of checking the dust map.

FIGS. 6 a-6 f represents an optical geometry of a lens that may be usedin accordance with a preferred embodiment:

FIGS. 6 a, 6 b and 6 c represent three lenses with the same, fixed,focal distance, with the same focal number, but with differentconstructions.

FIG. 6 d illustrates the concept of an exit pupil and the distance toit.

FIG. 6 e illustrates the intersections of the principal ray with windowsand image plane, which is preferably approximately the sensor plane.

FIG. 6 f illustrates the shift of one dust spec in comparison oa a shiftof another dust spec in an image.

FIGS. 7 a-7 g generally illustrate effects of dust on the creation of animage using a known optical system:

FIG. 7 a illustrates an influence of a dust particle located on theinput surface of the window on the beam converging towards the imagingpoint.

FIG. 7 b illustrates a side view of the rays as obscured by dust as afunction of the aperture.

FIG. 7 c illustrates a frontal projection of the rays as obscured by adust as a function of the aperture.

FIG. 7 d and FIG. 7 e illustrate a power distribution map of the samedust spec as manifested in different f-stops.

FIG. 7 f illustrate an effect of the change in focal length on the areaobscured by the dust.

FIG. 7 g illustrates the direction of movement of dust as a function ofthe change in focal length in relation to the center of the opticalaxis.

FIG. 8 illustrates an adjustment of the dust map based on aperture.

FIG. 9 illustrates an adjustment of a dust map based on focal length.

FIG. 10 illustrates a process of estimating based on empirical data thevalues of the parameters in the formulae that defines the change in dustas a function of change in focal length.

FIG. 11 illustrates a process of estimating based on empirical data thevalues of the parameters in the formulae that defines the change in dustas a function of change in aperture.

FIG. 12 illustrates a further process of in accordance with anotherembodiment.

BRIEF DESCRIPTION OF TABLES

Table 1 lists parameters in mathematical formulations of the opticalsystem.

Table 2 lists potential Extracted Lens Parameters.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Some Definitions

Dust specs: The preferred embodiment takes advantage of the fact thatmany images may have repetitive manifestation of the same defects suchas dust, dead-pixels, burnt pixels, scratches etc. Collectively, forthis specifications all possible defects of this nature are referred toin this application as dust-specs or dust defects. The effects of thosedust specs on digital images are referred to herein as dust artifacts.

Acquisition device: the acquisition device may be a a multi-functionalelectronic appliance wherein one of the major functional capabilities ofsaid appliance is that of a digital camera. Examples can be includedigital camera, a hand held computer with an imaging sensor, a scanner,a hand-set phone, or another digital device with built in optics capableof acquiring images. Acquisition devices can also include film scannerswith a area-capture CCDs, as contrasted, e.g., with a line scanmechanism. D-SLR: Digital Single Lens Reflex Camera. A digital camerawhere the viewfinder is receiving the image from the same optical systemas the sensor does. Many D-SLR, as for SLR cameras have the capabilityto interchange its lenses, this exposing the inner regions of the camerato dust.

A few parameters are defined as part of the process:

-   N Number of images in a collection.-   HIdp Number of images for occurrence to be high dust probability-   HSdp Number of recurring specs for label a region to be high dust    probability-   p(hsdp) Probability threshold for high confidence of a dust spec.-   N dp Number of images to determine that a region is not dust-   p(ndp) Probability threshold to determine that a region is not a    dust region.    Most likely Hidp <=HSdp-   I—is a generic image-   I(x,y) pixel in location x horizontal, y vertical of Image I-   pM is a continuous tone, or a statistical representation of a dust    map-   dM is a binary dust map created form some thresholding of pM.

Mathematical Modeling of the Optical System

Prior to understanding the preferred embodiments described herein, it ishelpful to understand the mathematical modeling of the camera opticalsystem. With this modeling, the preferred embodiments may advantageouslyutilize a single dust map for dust detection and/or correctiontechniques rather than creating a dust map for each instance of dustartifact having its own shape and opacity depending on extractedparameters relating to the imaging acquisition process. With an abilityto model the optical system and its variability, a single map maysuffice for each lens or for multiple lenses, and for multiple focallengths, multiple apertures, and/or other extracted parameters asdescribed in more detail below.

In order to study the shadow of an object on a plane, it is helpful toconsider the following:

-   -   the illumination of the object (the spectral and coherent        characteristics of the light)    -   the shape of the object (including its micro-geometry)    -   the reflection and transmission properties of the object    -   the relative position of the object and the plane

The case of interest for defining the model of dust as it appears on thesensor, is that of an object close to the image plane of a lens, whichis a complex optical system, with natural light illumination. The objectshape and its reflection and transmission properties are practicalimpossible to model because no any specific information on the dustparticle is available. On the other side, because the dust particles aresmall, it is very probable to have the same reflection and transmissionproperties on their surfaces. The distance between the dust and thesensor, which is the image plane is small and is in the order ofmagnitude of fraction of a millimeter.

Some Definitions as to notations are now provided: TABLE 2 Parameters inmathematical formulation of the Optical system. Pe - exit pupilposition, the distance between the sensor, and the Exit Pupil tw -thickness of the window, distance of dust to the image plane f′ - focallength of the objective f/# - focal number (u, v) - denote thecoordinate system of the input surface, having its origin in theintersection point with the lens optical axis. (x, y) - the dustposition on the image plane in the coordinate system in the image planewith the origin in the intersection point with the lens optical axis.I(x, y) - be the value of the irradiance in the image point (x, y) D -is the exit pupil diameter h₀ is the distance of the objects on the dustplane to the center of the optical path h is the distance form theobject to the center of the optical axis h_(k) is the distance of dustspec k on the dust plane to the center of the optical path

FIGS. 6 a, 6 b and 6 c represent three lenses with the same, fixed,focal distance, with the same focal number, but with differentconstructions. The first construction, in FIG. 6-a is the most common.The second in FIG. 6 b is specific for the lenses used in metrology.This type of lens is called telecentric in the image space. The third inFIG. 6 c construction is rarely found in optical systems, but notimpossible.

FIG. 6 d illustrates the concept of an exit pupil and the distance toit. The exit pupil distance, 644, is the parameter that defines thisproperty, and represents the distance from the secondary principalplane, 646, of the lens and the intersection of the axis of the lightcone and optical axis of the lens. The focal distance 644 represents thedistance form the secondary principal plane to the object plane.

In the case of the second type lens, as defined in FIG. 6 b, the exitpupil position is at infinity. In the first and third case, FIGS. 6 aand 6 c respectively, the intersections of the light cone with the dustplane are ellipses. In the case of the second lens type, FIG. 6 b theintersection of the light cone with the dust plane is a circle.

In the case of zoom lens, the exit pupil position can be constant (atinfinity), or can vary significantly, depending on the construction ofthe lens.

This information about the exit pupil is usually not public because, ingeneral, it is not useful for the common user of the photographic or TVlenses. However, this information can be readily available with somemeasurements on an optical bench. Alternatively, this information can beachieved based on analysis of dust in the image.

FIG. 6 e illustrates the Intersections of the principal ray with windowsand image plane, which is the sensor plane. This figure illustrates thevarious parameters as defined in Table 1. FIG. 7 a describes theinfluence of a dust particle (obscuration) located on the input surfaceof the window (obscuration plane), on the beam converging towards theimaging point.

FIG. 7 b illustrates the side view of the rays as obscured by a dust asa function of the aperture. FIG. 7 c illustrates the frontal projectionof FIG. 7-b, of the rays as obscured by a dust as a function of theaperture. FIG. 7 d and FIG. 7 e illustrates a power distribution map ofthe same dust spec as manifested in different f-stops, namely a relativeopen aperture f-9 for FIG. 7 d and a closed aperture, namely f-22 andhigher, in FIG. 7 e. Once can see that the dust spot as recorded withthe closed aperture, is much sharper and crisper.

The dust particle is situated in the window plane, and totally absorbingor reflecting the portion of the beam (S2) that normally reaches theimage point (P). Let I(x,y) be the value of the irradiance in the imagepoint (x,y) in the absence of the dust particle. Assuming that theenergy in the exit pupil of the optical system is uniformly distributed,then, in the presence of the dust particle, the I(x,y) will be reducedproportionally with the ratio between the area of the intersection ofthe light beam (S1) with the clean zone of the window, and the wholearea of the same intersection (Sc).

Remember that Sc is a function of the lens f-number (f/#) and windowthickness t_(w) So, the value of the real irradiance in the presence ofdust will be $\begin{matrix}{{I^{\prime}\left( {x,y} \right)} = {{I\left( {x,y} \right)}\frac{S1}{Sc}}} & (4)\end{matrix}$or, by taking into account that S1=Sc−S2 and S2=Area of intersectionbetween the dust particle and the illuminated area: $\begin{matrix}{{I^{\prime}\left( {x,y} \right)} = {{{I\left( {x,y} \right)}\left( {1 - \frac{S2}{Sc}} \right)} = {{I\left( {x,y} \right)}\left( {1 - \frac{S_{D\bigcap C}}{Sc}} \right)}}} & \left( {4\text{-}a} \right)\end{matrix}$Because the dust particle is small, the next assumption can be made:

For all the image points affected by a specific dust particle, the areaSc remains constant. In the case of the telecentric illumination, thisassumption holds true for all the image points.

With the assumption above, we can correlate the dust geometry, positionand shadow.

First of all, we now study the distribution of the irradiance on theinput plane of the window. The intersection of the conic beam with thisplane is generally an ellipse. The major axis of this ellipse is alongthe intersection of the plane mentioned above and the plane determinedby the image point and the optical axis of the lens.The minor axis is $\begin{matrix}{b = {t_{w}\frac{D}{P_{e}}}} & (5)\end{matrix}$whereas the major axis is $\begin{matrix}{a == {t_{w}\frac{D}{P_{e}}\frac{1}{\cos\left( {{arctg}\quad\frac{h}{P_{e}}} \right)}}} & (6)\end{matrix}$where D is the exit pupil diameter, so $\begin{matrix}{D = \frac{f^{\prime}}{f/\#}} & (7)\end{matrix}$In order to reach a complete formula of the irradiance distribution wenow stress the assumption that for all image points affected by a chosendust particle, “a” varies insignificantly Let (u₀,v₀) be the point ofintersection of the principal ray with the plane mentioned above.

So, the characteristic function of the illuminated zone will beC=C(u,v,u ₀,v₀,f′,f/#,P_(e),t_(w))  (8)

Let D(u,v) be the characteristic function of the dust Then, the centerof the cone intersection with the input plane of the window will begiven by (3).

Thus, $\begin{matrix}{S_{D\bigcap C} = {\int_{R}{\int_{xR}{{D\left( {u,v} \right)}*{C\left( {u,v,u_{0},v_{0},f^{\prime},{f/\#},P_{e},t_{w}} \right)}{\mathbb{d}u}{\mathbb{d}v}}}}} & (9) \\{{I^{\prime}\left( {x,y} \right)} = {{I\left( {x,y} \right)}\left( {1 - \frac{\begin{matrix}{\int_{R}{\int_{xR}{{D\left( {u,v} \right)}*{C\left( {u,v,u_{0},} \right.}}}} \\{\left. {v_{0},f^{\prime},{f/\#},P_{e},t_{w}} \right){\mathbb{d}u}{\mathbb{d}v}}\end{matrix}}{Sc}} \right)}} & (10)\end{matrix}$

-   -   which eventually, using the relation (3), yields $\begin{matrix}        {{I^{\prime}\left( {x,y} \right)} = {{I\left( {x,y} \right)}\left( {1 - \frac{\begin{matrix}        {\int_{R}{\int_{xR}{{D\left( {u,v} \right)}*{C\left( {u,v,{\left( {1 + \frac{t_{w}}{P_{e}}} \right)x},} \right.}}}} \\        {\left. {{\left( {1 + \frac{t_{w}}{P_{e}}} \right)y},f^{\prime},{f/\#},P_{e},t_{w}} \right){\mathbb{d}u}{\mathbb{d}v}}        \end{matrix}}{Sc}} \right)}} & (11)        \end{matrix}$        The terms (1+t_(w)/P_(e))x and (1+t_(w)/P_(e))y determine the        “movement” of the dust shadow with the change of the focal        distance and, implicitly, of the Pe, as explained later on in        equation 13.

In the case of the telecentric illumination

-   -   Pe is infinite,    -   u0=x and v0=y.    -   the ellipse becomes a circle

Qualitatively, as described quantitatively in equation 11, the fall off,or the diminished effect of the dust on an image varies as follow: Thedust is becoming less significant as the aperture becomes larger, or thef-stop is smaller and the pixels are inversely affected based on thedistance of the pixel to the periphery of the dust.

FIG. 7 f illustrates the adaptation between the dust map and the imagebased on the focal length of the lens. In this figure one canqualitatively see that the area covered by the dust spec will be shiftedas a function of the focal length.

Also, information about the spatial distribution and/or orientation ofdust particles within the dust map may be used to assist withdetermining probabilities that regions are either dust or non-dust. Asan example, one could expect that a global average of the orientation ofelongated dust particles would average to zero. However, at a morelocalized level, charged and/or magnetized dust particles can tend toalign with similar orientations due to local dielectric forces. Thiseffect would be partly dependent on the size, overall shape distribution(round or more elongated) and the overall distribution (uniform orclumped) of the dust regions in the dust map. This can tend to beparticularly helpful when a decision is to be made as to whether toeliminate certain regions. E.g., most of the dust may be aligned in aradial direction relative to the center of the sensor, and thenparticles in a concentric alignment are less likely to be dust. Giventhe definitions above, a more quantitative model can be deducted.$\begin{matrix}{h = {{h_{0}\frac{1}{1 + \frac{t_{w}}{P_{e}}}} \cong {h_{0}\left( {1 - \frac{t_{w}}{P_{e}}} \right)}}} & (12)\end{matrix}$because the ratio t_(w)/P_(e) is small. Note that Pe has in common casesa negative value.

Let (u,v) denote the coordinate system of the input surface, having itsorigin in the intersection point with the lens optical axis. Similarly,let (x,y) be the coordinate system in the image plane with the origin inthe intersection point with the lens optical axis. Thus, if theprincipal ray of the beam intersects the input window plane in the point(u,v), the point in the image plane on the same principal ray will beThe inverse transformation of (2) $\begin{matrix}{\begin{bmatrix}x \\y\end{bmatrix} = {\begin{bmatrix}{1 - \frac{t_{w}}{P_{e}}} & 0 \\0 & {1 - \frac{t_{w}}{P_{e}}}\end{bmatrix}\begin{bmatrix}u \\v\end{bmatrix}}} & (2)\end{matrix}$with the same assumption of a small tw/Pe ratio will be $\begin{matrix}{\begin{bmatrix}u \\v\end{bmatrix} = {\begin{bmatrix}{1 + \frac{t_{w}}{P_{e}}} & 0 \\0 & {1 + \frac{t_{w}}{P_{e}}}\end{bmatrix}\begin{bmatrix}x \\y\end{bmatrix}}} & (13)\end{matrix}$FIG. 7 j depicts how points will shift as a function of the proximity tothe optical center. Basically the farther the dust is form the center,the larger the displacement is. However, the direction and size of thedisplacements can be estimated. At the same time, the closer the dustspot is to the center, the more its shape will change.

There are a few parameters that are not published. The distance, Tw, ofthe dust to the image plane is typically fixed for the camera and asingle measurement can achieve that. The exit pupil distance may varydepending on the focal length of the lens, in particular for a zoomlens, the Exit pupil distance may not be most likely fixed, and can varyup to a telecentric mode where the distance is infinity.

However, such information can be empirically extracted by analysis ofimages taken by a known camera and lens, as illustrated below and inFIG. 7 e. In this case, knowledge of the effect of one dust spec can beused to calculated the shift for any other dust spec in the image.

In this figure a hypothetic zoom lens of 70 mm-210 mm is illustrated:

-   -   Pe-70 is the exit pupil position for the lens at 70 mm,    -   Pe-210 is the exit pupil location when the lens is at its        highest enlargement 210 mm.    -   k, is known dust    -   m is hypothetical dust    -   h_(k) is h for a specific dust particle k.

The knowledge of the shift of dust image (preferable near to one of thecorners of the image). The equation (12) $\begin{matrix}{h \cong {h_{0}\left( {1 - \frac{t_{w}}{P_{e}}} \right)}} & (12)\end{matrix}$

-   -   can be written for a specific dust particle (k) and underline        the dependence with the focal distance, $\begin{matrix}        {{{h_{k}(f)} = {h_{0k}\left( {1 - \frac{t_{w}}{P_{e}(f)}} \right)}}{or}} & \left( {12\text{-}a} \right) \\        {\frac{h_{k}(f)}{h_{0k}} = {\left( {1 - \frac{t_{w}}{{Pe}(f)}} \right){\quad\quad}{independent}\quad{from}\quad k}} & \left( {12\text{-}b} \right)        \end{matrix}$        So, if we know the evolution of the image of the “k” dust        particle we can find the evolution of the “m” dust image        $\begin{matrix}        {{h_{m}(f)} = {\frac{h_{0m}}{h_{0k}}{h_{k}(f)}}} & \left( {12\text{-}c} \right)        \end{matrix}$        The problem with this formula is again we don't know the        positions of the dust particles Writing the upper formula for a        given focal distance, say f₀, we have $\begin{matrix}        {\frac{h_{0m}}{h_{0k}} = \frac{h_{m}\left( f_{0} \right)}{h_{k}\left( f_{0} \right)}} & \left( {12\text{-}d} \right)        \end{matrix}$        So, finally $\begin{matrix}        {{h_{m}(f)} = {\frac{h_{m}\left( f_{0} \right)}{h_{k}\left( f_{0} \right)}{h_{k}(f)}}} & \left( {12\text{-}e} \right)        \end{matrix}$

This relation do not need the exit pupil dependence, the thickness ofthe window and the dust position. FIG. 7-g, illustrate the phenomenavisually. Based on the above formulae, and in particular 12-e and 13,one can see that the dust movement is not constant, as a function of thefocal length change, but is dependent on the distance of the dust to thecenter of the optical path. Qualitatively, the farther the dust is formthe center, the larger the displacement will be. In other words, dustcloser to the periphery of the image will have larger movement than thedust in the center.

Referring to FIG. 7-g which describes the movement of two hypotheticaldust spots, 790 and 794 having a different distance, 792 and 796respectively, to the center of the optical path 780. The movement ofeach pixel in always on the line between the pixel and the center of theoptical path. This is based on the linear relationship depicted inequation 13. Thus the movement of the dust edge of dust spec 794 isalong the lines depicted in block 799. Based on assumed coefficients forequation 13, dust spec moves be a fixed ratio${1 + \frac{t_{w}}{P_{e}}},$which can be positive or negative, which means move inwards towards thecenter or out of the center. In this figure, dust spec 794 will move tobecome 795 while dust spec 791 will move to become 791. because themovement is linear, because the distance of dust 794 to the centerdepicted by 796 is larger than the corresponding distance depicted byblock 792, the movement of the dust 794 will be larger.

Alternatively, given formula (13) and given an image where the dust canbe detected, one can calculate Pe as follows: $\begin{matrix}{{\frac{x}{u} = {\frac{y}{v} = {1 + \frac{t_{w}}{P_{e}}}}}{or}} & \left( {13\text{-}a} \right) \\{P_{e} = {{\left( {1 + t_{w}} \right) \times \frac{u}{x}} = {\left( {1 + t_{w}} \right) \times \frac{v}{y}}}} & \left( {13\text{-}b} \right)\end{matrix}$

In summary of the mathematical model of the optical system for dust, inthe general case, the way the image is affected by the dust depends on:

-   -   exit pupil position Pe    -   thickness of the window tw    -   focal length of the objective f′    -   focal number f/#    -   the dust position on the image plane (x,y)

This can be calculated by knowing the optical system's components:

-   -   a) the window thickness (tw),    -   b) the function Pe(f)    -   c) and    -   d) the coordinates (u,v) of the dust if we want to determine the        dust image position on the image plane

The main workflow of detecting and removing the dust from an image isillustrated in FIG. 1. The preferred embodiment is activated in fourdifferent cases. In general, this preferred embodiment works forremoving dust form a collection of images having the same acquisitiondevice. Specifically, a user may acquire a picture on her digital camera(as illustrated in Block 101). Alternatively (102), a user may open asingle image on a external device such as a personal computer, open(103) a folder of images on an external device or open a collection ofimages on a digital printing device (104).

The preferred embodiment then extracts the shooting parameters (120).Such parameters include, but not limited to data about: Camera name,Lens brand, lens type, focal length at acquisition, aperture range,aperture at acquisition.

In addition, some parameter, in particular on the lens and the cameramay be also stored in the device, which is the acquisition device suchas the digital camera or the processing device such as the personalcomputer or digital printer. Such information, which may includeparameters such as exit pupil, exit pupil distance regarding the lens,or distance of dust to sensor for the camera.

A table with such data may look like: TABLE 2 Extracted Lens ParametersField Example of data Category Lens Nikon lens Manufacturer Lens Type AF24 mm-45 mm f2.8-f3.5 lens Focal Length   38 mm Acquisition dataAperture f-16 Acquisition data Dust distance 0.156 mm Camera data Exitpupil   19 mm Lens data Exit pupil   230 mm Lens data distance

The dust map may also include meta-data that are different the list ofextracted parameters. Moreover, that which is described as beingextracted parameter dependent or encoded with extracted parameter valuedata or based on a value of an extracted parameter can be broadened toinclude other meta-data than just the extracted parameters listed inTable 2. For example, certain meta-data are dependent on parametersexisting at the time of acquisition of the image, and can becamera-specific or not. The amount of ambient light available willdepend on whether there is artificial lighting nearby or whether it is acloudy day. Discussion of meta-data as it relates to image acquisitionis found in more detail at U.S. patent application Ser. No. 10/608,810,filed Jun. 26, 2003, and is hereby incorporated by reference.

In the case that the system deals with multiple images (as defined in102, 103, and 104), the algorithm describes a loop operation on allimages (110). The first step is to open a dust map (130). If non exists(132) the system will create a new dust map (200) as further describedin FIG. 2. In the case that the system has a few dust maps (130) thesoftware will try to correlate one of the maps (300) to the image. Thisprocess is further illustrated in FIG. 3. In particular this correlationrefers to the adjustment of the shooting conditions to some acceptedstandard. The acquisition conditions in particular refer to the apertureand the focal length. The correlation process is interchangeable and canbe done by adjusting the image to the map or adjusting the map to theimage. In some cases both acquired image and dust map should be adjustedto some common ground. Such an example may happen when the ma iscalculated based on a aperture that the lens does not reach or adifferent lens than the one used wit different optical configuration.Alternatively, such as in the case of a new lens, this process (300) ascalled by 130, may be used to adjust the map onto a new map and fromthat stage onwards continue with a single map.

If no dust map corresponds with the image (140), a new dust map iscreated (200). When a dust map does correspond (140) to the image, thepreferred embodiment checks if the dust specs as defined in the dust mapare of high enough confidence level to being dust regions (150). Thestatistical decision as to the way such confidence level is calculatedfor the dust map in general and for the individual dust specs, isfurther discussed in FIG. 2 and FIG. 3. If the confidence level is low,the image is added to the updating of the dust map (200). If after theimage is added, the confidence level is high enough (152) the softwarecontinues to the dust removal process (160). Otherwise, the softwareprogresses to the next image (170).

For example, a dust map is considered valid after 10 images areanalyzed, and a dust spec is considered valid after a dust is detectedin 8 images. In this case, after analyzing 9 images, the software maycontinue to the stage of updating the dust map (200) but upon completion(10 images) there is a valid dust map, and the software will continue tothe correction (160) stage. If however the loop (110) is only on its1^(st) to 8^(th) image, no correction will be done.

As an additional embodiment, images can be corrected retroactively afterthe dust map reached high enough confidence. This can be used for batchprocessing or off line processing or processing where the information isgathered in parallel to the needed correction. In this embodiment, whenthe confidence level is not enough (152, NOT-YET) the images, a pointerto them, or a list of them, or a representation of them, that were usedto create the dust map are stored in temporary location (154) and whenthe dust map is ready (151-YES), the software will retroactively removethe dust from all those images (156). In this fashion, all images,including ones that originally did not hold sufficient statisticalinformation, may be corrected.

Referring to the dust detection and correction process (160). Thisprocess may be executed as a loop on every dust spec in the dust map,with a stage of detection and correction (400, followed by 166 and 168).Alternatively, the process can be implemented where all dust specs aredetected first (162) and then all dust specs are corrected (164). Thedecision as to the sequence of operations varies based on implementationcriteria such as what regions of the image are in memory, and should notaffect the nature of the preferred embodiment. As part of the detection,the software also performs a self testing (500) where each new imageintroduced is compared to the dust map. This process is further definedin FIG. 5. The importance of this stage for each dust spec and for thedust may, is that in this manner, if the dust situation changes, such asa single spec moving aground or the camera being serviced), the softwarewill immediately detect the change and re-validate the dust map. Asdescribed above, the validity test (500) can be implemented on a dustspec by dust spec or on the full dust map.

Referring to FIG. 2-a where the Dust map creation and updating isdefined: This process can receive a collection of images as defined byFIG. 1. blocks 108 and 109) or one image at a time is refereed to thisprocess, as defined by FIG. 1. block 110. If the function is called witha single image (220—SINGLE IMAGE) the image is directly provided to thecalculations (270). If multiple images are provided (240 MULTIPLEIMAGES), then an initial step is to define if there are more than enoughimages for defining the map. This step is designed to optimize thecreation process for dust in case of large amount of images.

The sequence of the images that are to be referenced based on theoriginal collection of N images as defined in FIG. 1 blocks 102, 103 or104. The sequence of images is based on a few criteria such as: givingmore weight to images shot last, and if images are shot in a relativelysmall time frame, allocate the sequence is large distances to try andassure minimal repetitiveness between similar images that may have beentaken of the same object with little movement. The sequence will not belimited to the number of images (HIDP) because it may well be that someregions will not have enough data in them to evaluate the dust. This mayhappen in cases where part of the image is very dark in some of theimages.

As an example: if N (number of images in a selection)=30; and HSdp(number of images needed to determining map)=10; and all images wereshot in a space of an hour; then a potential sequence may be:

-   -   30, 27, 24, 21, 18, 15, 12, 9, 6, 3, 29, 26, 25 . . . 2, 28, 25        . . . 1

Alternatively if the same 30 images were taken over a period of a monthit may be beneficial to select images sequentially (last one shot is thefirst to be calculated):

-   -   30, 29, 28 . . . 20, 19 . . . 2, 1        And in some cases this process of sampling the series (270) may        also decide not to use images that are taken too long from the        last image. In this case, for example if image 1-15 were taken        in July and 16-30 were taken in November, this process may limit        the new map to 16-30 or even create two different maps, one for        images 1-15 and the other for 16-30.

In a different criteria, the parameters as extracted from the imageswill determine the sequence and the number of dust maps that are to becalculated. For example if a folder contains N=30 images, where 15 weretaken with one camera and 15 with another, the sampling step (270) maycreate two map sets.

Another criteria for creating a new set or checking for new dust is thetype of lens. If a lens is changed, it means that the CCD-cavity wasexposed to potential new dust. This may trigger a new set of images tobe looked at. It may also be an indication that the camera was serviced,or that the photographer cleaned the camera. Of course, if there is aparameter that defines when a camera was serviced, this will trigger thecreation of a new dust map.

Those familiar in the art may be able to determine the rightcombinations of creating this sequence based on the nature of the dust,the camera and the lens. The next loop (270-271) defines the marking ofeach region and the addition of the region to the dust map if it is notalready there. There are three type of regions in an image, the firstare images with sufficient information to detect whether they are ofdust nature. As an example, dark regions surrounded by a lightbackground. Other criteria may be regions with a relatively small colorsaturation. The second group are regions that are definitely non-dust.Such regions are for example all clear, or of high color saturation.Other regions are inconclusive such as a very dark segment of the image.In this case, it will be hard to detect the dust even if it was part ofthe image.

Alternatively when looking for over exposed or “dead pixels” thecriteria may be reversed, if the pixels appear as a white spec in theimage.

The criteria may be also a function of the acquisition parameter. Forexample an image with a open aperture may all be marked as in-decisive,because the dust may not appear on the image. Regions that arepotentially dust are marked (292) and then added to the dust mask (294).The addition may be the creation of a new dust spec on the map or themodification of the probability function or the confidence level counterfor the region. Regions that are most likely non-dust are marked (282)and then added to the dust mask (284). The addition may be the creationof a new dust spec on the map or the modification of the probabilityfunction or the confidence level counter for the region. The additionsof the regions needs to be normalized to the shooting conditions asdefined by the dust map (300) if this step was not performed prior toentering this function, as optionally defined in FIG. 1.

This loop continues over all regions of the image (271). Alternatively(272), each region is compared (500) with the map dust to se if there isno case where the monotonicity is broken, i.e. a region that was of highprobability to be dust is now non dust. FIG. 2-b describes an alternateembodiment of the dust creation process. Block 1210 describes thepreparation of the dust map which includes either opening an existingmap 1216 or creating a new one 1214, where all pixels as a startingpoint are non dusst or WHITE. After the map is correlated to theshooting conditions, 300, a dust map I-dM is created in block 1220.

All pixels in the image, 1222, receive the following values 1224 basedon the luminance value of the pixel: Case

-   If luminance is less than DARK_THRESHOLD, then I-pM    (X,Y)=MAYBE_DUST;-   If luminance is greater than WHITE_THRESHLOD, then I-pM(x,y)=WHITE    PIXEL;-   OTHERWISE I-pM(x,y)=DON'T_KNOW.

Once all pixels are analyzed, 1220, they are then clustered, 1296 intodust regions or dust specs in a I-pM dust map. The next step is tocreate a dust map dM which is continuous based on the value of theindividual pixels. In the final stage 1250, the dust map is threshold bythe predetermined value THRESHOLD to create a binary mast.

FIG. 3 illustrates the process of correlating the image to a defaultsettings of the dust map. This process defines correlating the image othe dust map, the dust map to a new dust map of the dust map to themage. In particular this correlation refers to the adjustment of theshooting conditions to some accepted standard. The acquisitionconditions in particular refer to the aperture and the focal length. Thecorrelation process is interchangeable and can be done by adjusting theimage to the map or adjusting the map to the image. In some cases bothacquired image and dust map may be adjusted to some common ground. Suchan example may happen when the ma is calculated based on a aperture thatthe lens does not reach or a different lens than the one used witdifferent optical configuration. Alternatively, in case of a new lens,this process (300) may be used to adjust the map onto a new map and fromthat stage onwards continue with a single map.

To begin with, the dust map is being loaded (112) and the default dataon which the map was generated is extracted (310). Such data may includethe lens type, the aperture and the focal length associated wit thedefault state. In concurrence, the information form the acquired image(304) is extracted (320) and compared to the one of the dust map.

A explained in the mathematical model of the optical system, the twomain adjustments between the dust map and the image are based on focallength, and on aperture, each creating a different artifact that shouldbe addressed. Knowledge of the phenomena may assist in creating a betterdetection and correction of the dust artifact. Alternatively, in aseparate embodiment, analysis of the image and the modification of thedust as changed by aperture and focal length, may provide the necessarymathematical model that describes transformation that defines thechanges to the dust as a function of change in the lens type, the focallength and the aperture.

The mathematical distortion of the dust as a function of the aperture isillustrated in FIGS. 7 a-7 f. The geometrical optics illustrations ofthe above phenomena are depicted in FIGS. 6 a-6 e. Referring to FIG. 3,after extracting the data, the following step is modification of the mapand or the image based no focal length (900), and based on aperture(800). The following steps are further defined in FIG. 9 and FIG. 8respectively.

Following the too step (800 and 900) the Image and the Dust Map areconsidered to be correlated. The correlated map cM is no longer binarybecause it defines both the shift and the fall off which is continuous.FIG. 400 defines the process of detecting and removing the dust from theimage. The input is the image I is loaded, if it is not already inmemory (404) and the correlated dust map is cM is loaded (402) ifalready not in memory.

The process of detecting and removing the dust is done per dust spec.This process is highly parallelized and can be performed as a singlepath over the image, or in strips. The flexibility of performing theoperation on the entire image, or in portions, combined with thecorrelation or as a separate process, enables a flexible implementationof the algorithm based on external restrictions defined by the hardware,the run time environment, memory restrictions and processing speed.

As defined and justified by the physical phenomena, the method ofcorrecting dust artifacts is defined based on two different operations.The former is the retouching or the in-painting here regions with nodata (420), or data that is close to noise to be recreated as defined in430 and later on in FIGS. 7 e, 7 f. The second portion of the correctionis based on the tapering degradation of surrounding pixels as a functionof the aperture, as illustrated in FIGS. 7 b,7 c,7 d. Referring to theimage enhancement portions, where the data exists but most likely not inits full form, due to the fact that some of the dust affects the qualityof the image, but some regions still contain some data (420), an imageenhancement algorithm is performed on the pixels (430). In a simplifiedembodiment (432), assuming that the optical model is simplified to adegradation in the overall brightness as defined by the OPACITY, theenhanced pixels will receive a value inversely related to the OPACITYi.e. 9432)${I^{\prime}\left( {x,y} \right)} = \frac{I\left( {x,y} \right)}{OPACITY}$

To maintain a check of the validity of the model, the pixels beforebeing operated on may be send for validation (500) as described in FIG.5. The second portion of the image correction is the restoration or theinpainting (460). In this case, the area behind the dust has no relevantdata to enhance, or if there is, this data is relatively close to theoverall noise level and thus can not be enhanced. Therefore, there is adesire to in-paint every pixel based on the analysis of the surroundingregion to the dust (470). In the case where the enhancement regions asdefined in block 430 are of good quality, those pixels as well maycontribute to the in-paining. Otherwise, they are excluded form thecalculations. The decision may rely on the distance of the pixels to theOPAQUE ones, of the minimum level of OPACITY that was used to restorethe pixels in block 430. The larger the OPACITY, the smaller such pixelsmay be relied on in the inpainting process.

FIG. 4 b illustrates the a certain embodiment of the in-paintingprocess. In general, each pixel in the obscured region, 480, is to befilled, 482, based on its surrounding pixels. In this specificembodiment, the pixels are filled in based on information of thesurrounding non affected pixels. This specific algorithm takes intoaccount that the pixels closet to the periphery has a better chance tobe anticipated by the external pixels. Therefore the process is doneform the outside inwards.

This process of spiraling inwards is also depicted in FIG. 4 d. In it,given a dust spec 1400 in a grid 1410 the dust is taken a digital form,of the surrounding bounding area 1420. External pixels such as 1430 arenot part of the dust region, while internal ones such as 1450 are. Analgorithm that works on the periphery moving inwards as defined in FIG.4 b blocks 470 is depicted in FIG. 4 d as follows: In the first roundall peripheral pixels, numbered form 1 to 20 are being operated on asdefined in FIG. 4 b, block 474. After that, all of the above mentionedtwenty pixels, as defined in FIG. 4D Block 1460 are removed, accordingto block 476 of FIG. 4 b, form the region leaving a smaller dust spec ofshape 1470. This modified dust spec, has a different set of peripheralpixels numbered 21-31. After removing those pixels by the in paintingprocess, block 467, a smaller dust kernel as depicted in 1480 is leftwith only three pixels 34, 35 and 36.

The process of filling in the pixels need not be spiral. In a differentalternative, the process follows the pattern of the region surroundingthe dust. For example lines and edges, or other high frequencyinformation are being filled in prior to filling in the smooth regions.This approach will prevent unnecessary blurring due to the in-paintingprocess.

A preferred embodiment the criteria for the chosen value can be based onmarinating the overall gradient around the pixel, 462. The justificationis that in the case of a steep edge, it is not advisable to copyinformation for the surrounding pixels with the edge value. By doing so,the in painting process will maintain any high frequency information inthe picture such as lines and edges.

An illustration of that is given in FIGS. 4 c, 4 d and 4 e. Referring toFIG. 4 c, the same dust 1400 as in FIG. 4 b with the bounding box 1420on a grid 1410 is obscuring a picture including high frequency data suchas the letter A, 1440. By taking a small section of based on 9 pixels asdepicted in FIG. 4 d pixel 1490 is surrounding by 8 pixels 1491, 1492,1493, 1494, 1495, 1496, 1497, 1498.

In a simplification, each pixel receives the average value between theblack and white regions. This is depicted in FIG. 4 e. In this case, thepixels 1490 and its surrounding 8 pixels 1491, 1492, 1493, 1494, 1495,1496, 1497, 1498. have a digital value of 21, 220, 250, 245, 253, 180,145 and 35 respectively. It is clear that the best value of pixel 1490will come form its surrounding pixels with the smallest gradient.Because in practice the value of pixel 1490 can not be determined, butis the center of the in painting, the value of the gradient will bebased on the value of the gradients of the pixels around it andextrapolated to this pixel.

In this specific example, the differential horizontal gradient betweenpixels 1497 and 1493, is the largest while the gradient verticalgradient of 1495 and 1491 will most likely be the same. Therefore, apreferred value of 210 will be based on the extrapolated average of thetwo gradients of its top and bottom pixels.

FIG. 500 describes a tool that may be used in various stages of theprocess, namely quality control. This process can be called from themain workflow (FIG. 1) the creation of the dust map (FIG. 3) or theimage restoration and enhancement (FIGS. 4 and 4-a respectively). Thepurpose of this took is to perform the necessary quality check to assurethat pixels will not be wrongly classified, nor mistakenly correctedwhen they should not.

The quality control test can include: For a pixel in an image whichbelongs to a potential dust (510). In case the image was not yetcorrelated with the dust (300) this process should then be done. Thenext step is to check whether the pixel belongs to a dust region or not.For example, in a small aperture the region behind the dust should beclose to if not totally obscured. If not (530 NO), there is a remainingdesire to revive the dust map (550) or the image does not correspond tothe dust map. This can happen when the image is form a different time,or from a different acquisition device. In this case, the software willcreate a new dust map (200) to correspond to the new dust or lackthereof, or inform the user that it can not correct this image. In casewhere there is not enough information (530 MAYBE), there is noconclusive evidence to reject the correlation and thus the processcontinues (580) with no action. In the case that the image displaysinformation that may concur with the dust map, the software may continue(580) or prior to that, enhance the likelihood in the dust map (200)that the region is indeed dust. FIG. 8 describes the adjustment of thedust map to the image acquisition parameters based on the aperture.Simplifying, the closed the aperture is, the crisper and more noticeablethe dust is. In other words, for example images taken with an f-stop off-32, the dust will be very prominent and opaque, while the same imagetaken at f-2.8 may display no visual degradation of the image. Thecorrection of the image should take that information into account, toprevent over correction of the dust artifact.

The acquisition information and the corresponding dust map default setupare extracted in blocks 326 and 312 respectively. Then, for each dustspec in the image 810, the size of the region that is still obscured bythe dust is calculated, as defined by mathematical model. In some cases,when the aperture is very open, this region may decline to 0. In others,where the aperture is still very close, the size may be close to thesize of the dust. Alternatively, This step, 820, may be done as apreparation stage, and kept in a database, which can be loaded.

The process then splits in two. The fully obscured region will be markedin 834 pixel by pixel 832 in a loop 834, 835 and will be treated by thein-painting process as defined in FIG. 4-a. A semi opaque dust map, iscreated in the loop 840, 841 for each pixel. Each of the pixels 842, isassigned an PCAPITY value 844, based on the mathematical model asdescribed previously in FIGS. 7 a-4 d. The dust spec that is onlypartially attenuated will go through a inverse filtering of theinformation already there, as described in FIG. 4 block 430, with aspecific embodiment in block 432. The process of the inverse filteringmay take into account the signal to noise ratio to avoid enhancing datawhich is not part of the original image. For example, the region aroundthe dust may have a over-shoot similar to a high pass filter, which maymanifest itself in the form of an aura around the dust. This aura shouldto be taken into account before enhancing the regions.

FIG. 9 describes the adjustment of the Dust Map based on the Focallength, and the specific lens. The scientific background is explained inFIGS. 7 e-7 f. As described before, the shift of the dust spec as afunction of focal length for a specific lens is a function of equation(13 the thickness of the window—tw, which is constant for a given cameraand exit pupil position—Pe which varies based on the lens system and thevariable focal length in case of a zoom lens. Given an image and a dustmap, the pertinent information is loaded, as described in FIG. 3, namelythe focal lens and lens type of the camera, 326, the focal length andlens type in the dust map 312 and the camera distance of dust to thesensor 318.

The process then goes through all knows dust specs in the image 910 andcalculates the shift of the dust as explained in FIGS. 7 a-7 e. Thecoordinates of the pixel are calculated from the center of the opticalpath, 922, and the shift is calculated 924. Alternatively to goingthrough each pixel, in order to speed the process, only the periphery ofthe dust spec can be calculated and the rest will be filled in.Moreover, because the shift is a function of the location (x,y) of thedust, in the case where dust is far enough from the origin, the dustshape will not change. It is then sufficient to calculate only a shiftof a single point and displace the entire dust spec accordingly. Theregions, which are only partially attenuated due to the change inaperture, may be calculated in this stage 940, 941, if calculatedalready as illustrated in FIG. 8, or alternatively, the displacement canbe calculated first as explained in blocks 942,944.

In some cases, it is impossible to get the data on the exit pupil, northe distance the dust is from the sensor. Such cases may be when theapplication has no a-priori knowledge of the camera or the lens that wasused.

It is still possible in those cases to create a reasonable dust map, byempirically reconstruction the parameter based on analysis of theimages, the shift in the dust and the falloff of the dust edges. Suchtechniques are defined in FIGS. 10 and 11 defining the process ofestimating, based on empirical data, the values of the parameters in theformulae that defines the change in dust as a function of change inaperture.

This process can be useful in the case that the software has no a-prioriknowledge of the extracted parameters of the camera or lens that areused to convert the acquired image and the dust map image into the sameknown space. In an alternative embodiment, this technique can also beused to extract optical parameters, similar to methods done in acontrolled laboratory settings such as using an optical bench of a lensregardless of the desire to remove dust artifact from an image.Alternatively this process can be used with partial knowledge, such asthe shift of a single dust to evaluate the transposition of all dustspecs in the image.

FIG. 10 Defines the process of creating the mathematical formulae basedon empirical data to parameterize the change in dust as a function ofchange in focal length. In general, a proposed embodiment relies on thefact that when a dust if found, a pattern matching can be applied tofind the shift in the dust. Based on this information,$\frac{t_{w}}{P_{e}}$can be calculated as defined in Equation 13-a. If t_(w) is known then Pecan be calculated as recited in equation 1-b.

Specifically, in a preferred embodiment, an image is acquired, 1010 anda dust map is calculated 1012. A second image is captured, 1030 with adifferent focal length than the first image, and a dust map iscalculated 1032. The process repeatedly tries to find two dust spots inthe two dust maps 1040. If no dust specs are correlated the process isrepeated for consecutive images, 1020.

The process of finding dust specs is calculated by applying a localcorrelation to each of the dust specs. Preferably, based on Equation13-a, the further the dust is from the center, the better the precisionis.

When two specs are determined to be from the same dust spec thedisparity between the specs is calculated 1042. The ration between theshifted pixels is calculated. This ratio is the numerical empiricalestimation of t_(w) in equation 13-a. Moreover, if the distance of thedust to the sensor is known, the Exit pupil of the lens can becalculated based on the same equation.

FIG. 11 Defines the process of estimating based on empirical data thevalues of the parameters in the formulae that defines the change in dustas a function of change in aperture The process is similar to the onedescribed for estimation of the focal length, albeit the fact that theparameters calculated are different.

Specifically, a first image is acquired, 1110 and dust is detected, 1120in this image. If the image is not appropriate for detecting the dust,or if the probability is low for the dust regions, this image isrejected for this purpose and another image is chosen. For the empiricalcomparison, a second image is captured 1140, or alternatively a set ofimages 1130, all with varying aperture, to enable acceptable samplingset. The process then looks for a detected dust region with high levelof probability. In the case that all other parameters are similar exceptthe aperture, the process can search for dust regions in the samecoordinates that the original image dust regions were found. The dustregions of the two or more images are correlated, 1160. The processcontinues for a sufficient amount of dust regions, 1168, which in somecases can even be a single one, and sufficient amount of images, 1169,which can also be, depending on the confidence level, a single image.Once the dust regions are correlated 1160, the fall off due to thechange of aperture is calculated, 1172, on a pixel by pixel basis, 1170,for every pixel in a dust region, 1179. Based on this information, thefall off function is calculated. 1180. In a preferred embodiment, thefall off function is determined as a function of the distance of a pixelfrom the periphery of the dust, as well as the aperture.

Alternatively, the dust specs may also be determined by trying tocorrelate the dust spec in the map to the acquired image. Suchcorrelation can be performed within a reasonable window size or even onthe diagonal line between the dust and the center of the optical path,in the direction that the dust may move. By gaining knowledge on themovement of a single dust spec, as explained in formulae 13 a-13 d, allother dust specs shift can be determined.

It is also possible to determine whether the camera should be physicallycleaned based on analysis of the dust in the camera and the specificdust patterns. This illustrated in the flowchart of FIG. 12. An examplewill be that certain width of dust will not allow correct in-paintingbased on the surround. Another example will be the overall number ofdust specs or the overall relative area that the dust covers. The inputfor such analysis is a dust map, 1200. This map can be similar to thedust map generated in block 200, or any other representation of a dustmap or a calibration map, generated automatically or manually by thephotographer. Such analysis need not be performed for every image. Aprocess, 1210 determines whether to perform such analysis. Examples totrigger this process are the time since the last analysis, change oflenses, which may create the introduction of dust, or message form thequality control of dust map, as defined in FIG. 5 block 500, that thedust has shifted or that the dust no longer corresponds to the map. Ingeneral, the any changes in the dust structure may be a justification totrigger the analysis process. When no analysis is desired, 1212, theprocess terminates. Otherwise, the analysis is performed, 1220. Theanalysis is performed for each dust spec individually, 1130, and thenthe results are accumulated for the whole image. For each dust region,1140, some parameters are being extracted, including, but not limited toArea of dust region; Maximum width of dust region; the distance betweendust region and neighboring regions; movement of dust region form lastanalysis; occurrence of new dust specs since last analysis; etc.Following this analysis, the results are summarized to include such dataas—the overall area of dust, in size and in percentage for the entireimage; the largest width of dust; the largest area of dust spec; changesin area of dust since last analysis and changes of dust particles sincelast analysis. In an alternate embodiment, either automatically or basedon the photographers preference, this analysis may be displayed or savedin a log file for future reference, 1221. Each of the aforementionedcriteria may have a acceptability threshold value. This threshold valueis determined empirically by the manufacturer in terms of the maximumacceptable dust that can be corrected in software. Alternatively, thisdata may be adjusted to by the photographer based on her tolerance levelto dust. If any of the parameters exceeds an acceptable threshold, theuser is informed, 1290, that the camera should be manually maintainedand cleaned up.

Alternatively, 1285, this same process may be used as a tool to informthe user of changes in the dust. Such information is particularlyimportant in the case that the dust correction algorithm is based on thecreation of a calibration image. In this case, the analysis will be usedto inform the user that a new calibration image should be acquired tosupport the correction of dust in future images.

Alternatively, the process of analysis described above may also beincorporated in a maintenance procedure, where the camera, after beingcleaned up will perform an auto-test, as described in FIG. 12, to verifythat the camera now is indeed clean. In such cases, the thresholdparameters are of course substantially more restrictive and demanding,to assure high quality maintenance.

There are many alternatives to the preferred embodiments described abovethat may be incorporated into a image processing method, a digitalcamera, and/or an image processing system including a digital camera andan external image processing device that may be advantageous. Forexample, an electronic circuit may be designed to detect maximum andminimum dust detection probability thresholds while acquiring pixels inan image (see also U.S. Pat. No. 5,065,257 to Yamada, herebyincorporated by reference). Such a circuit can produce signals that maybe used in processing to eliminate regions, which lie outside theexpected range of signal values due to the presence of dust particles orsimilar optical defects, or alternatively to accept, maintain oreliminate regions as dust artifact regions based on whether aprobability determination exceeds, high or low, a certain threshold orthresholds. A technique may be used to detect and provide a remedy forthe effects of dust on a digitally-acquired image (see also U.S. Pat.No. 5,214,470 to Denber, hereby incorporated by reference). An image maybe recorded within a digital camera or external processing device suchas may be signal coupled with a digital camera for receiving digitaloutput image information or a device that acquires or captures a digitalimage of a film image. The digital image may be compared with the filmimage, e.g., through a logical XOR operation, which may be used toremove dust spots or blemishes picked up in the acquisition of thedigital image from the film image.

Multiple images may be processed and stationary components, which arecommon between images, may be detected and assigned a high probabilityof being a defect (see also U.S. Pat. No. 6,035,072 to Read, herebyincorporated by reference). Additional techniques, which may be employedto modify defect probability, may include median filtering, sample areadetection and dynamic adjustment of scores. This dynamic defectdetection process allows defect compensation, defect correction andalerting an operator of the likelihood of defects.

Dark field imaging may be employed to determine the location of defectsin digital images from digital cameras or film scanners (see U.S. Pat.No. 5,969,372 to Stavely et al., and US patent application 2001/0035491to Ochiai et al., each hereby incorporated by reference). A normalimaging of a object with normal illumination may be followed by a secondimaging using different wavelengths, e.g., infrared illumination. Dust,fingerprints, scratches and other optical defects are typically opaqueto infrared light. Thus the second image produces an image with darkspots indicating the position of dust particles or other defects.

A process may involve changing any of a variety of extracted parameters(see elsewhere herein), angle of sensor relative to image plane,distance of image plane or sensor from dust specks (e.g., on window ofsensor), etc., and imaging a same object with the digital camera. Acomparison of the images reveals with enhanced probability the locationsof dust artifact. In a camera application, the unique location of theactual dust relative to the object and to the image plane provideinformation about extracted parameter-dependent characteristics of dustartifact in the images. The analysis for the digital camera applicationdepends on the “transmission”-based optical parameters of the system,i.e., the fact that light travels from a scene through the camera lensand onto the camera sensor, and not involving any substantial reflectiveeffects. It is possible to make determinations as to where the dustactually is in the system by analyzing multiple images taken withdifferent extracted parameters, e.g., on the sensor window, or in animage of an original object which itself is being images such as in filmimaging.

In a scanning application, this technique can be use the face that aspeck of dust will cast a shadow of a different color, geometrylocation, etc. with changes in extracted parameters, e.g., with adifferent color with increasing elongation of the shadow for eachparallel row of pixels (a “rainbow” shadow, as it were). Multiple scanstaken from various angles of illumination may be employed to produce animage which identifies dust defects from their shadows and the colorsthereof (see U.S. Pat. No. 6,465,801 to Gann et al. and US patentapplications 2002/0195577 and 2002/0158192 to Gann et al, herebyincorporated by reference). A linear scanning element moves across adocument (or the document is moved across the scanning element) and animage of the document is built up as a series of rows of pixels. Thisdiffers from the physical configuration of a camera in which a shutterilluminates a X-Y sensor array with a single burst of light. In bothcases, though, dust may lie close to the imaging plane of the sensor.

Technique may be applied as part of a photofinishing process toeliminate blemishes on a film image obtained by a digital camera (seealso US patent application 2001/0041018 to Sonoda, hereby incorporatedby reference). Such techniques may import previous acquired informationabout defects in images from a blemish detection procedure. A techniquefor correcting image defects from a digital image acquisition devicesuch as a digital camera may involve repeated imaging of an object orother image, where each successive image-acquisition involves differentproperties or extracted parameters or meta-data related properties, suchas variable angles of incidence or variable lighting or contrastparameters, and the results of these repeated scans may be combined toform a reference image from which defect corrections are made (see alsoUS patent application 2003/0118249 to Edgar, hereby incorporated byreference).

A decision on whether a defect in a image acquired by a field-baseddigital camera is to be corrected or not may be based on a balancing ofconsiderations. For example, the likely damage to surroundingdefect-free portions of the image may be balanced against the likelihoodof successfully achieving correction of the defect.

Image processing means may be employed where the detection or correctionof defects in a digital image may be based solely on analysis of thedigital image, or may employ techniques directly related to the imageacquisition process, or both. Anomalous image regions may be determinedbased on the difference between the gradient of an image at a set ofgrid points and the local mean of the image gradient (e.g., see U.S.Pat. No. 6,233,364 to Krainiouk et al., hereby incorporated byreference). Such technique can reduce the number of false positives in“noisy” regions of an image such as those representing leaves in a tree,or pebbles on a beach. After determining an initial defect list by thismeans, the technique may involve culling the list based on a one or moreor a series of heuristic measures based on color, size, shape and/orvisibility measures where these are designed to indicate how much ananomalous region resembles a dust fragment or a scratch.

Techniques and means to correct scratches in a digitized images mayemploy a binary mask to indicate regions requiring repair or noiseremoval, and sample and repair windows to indicate (i) the regionrequiring repair and/or (ii) a similar “sample” area of the image (seealso U.S. Pat. No. 5,974,194 to Hirani et al., hereby incorporated byreference). Data from a sample window may be converted to a frequencydomain and combined with frequency domain data of the repair window.When a low-pass filter is applied, it has the effect to remove thesharp, or high-frequency, scratch defect.

Techniques and means of detecting potential defect or “trash” regionswithin an image may be based on a comparison of the quadraticdifferential value of a pixel with a pre-determined threshold value (seeU.S. Pat. No. 6,125,213 to Morimoto, hereby incorporated by reference).The technique may involve correcting “trash” regions within an image bysuccessively interpolating from the outside of the “trash” region to theinside of this region.

Techniques and means to automate the removal of narrow elongateddistortions from a digital image may utilize the characteristics ofimage regions bordering the distortion (see also U.S. Pat. No. 6,266,054to Lawton et al., hereby incorporated by reference). User input may beused to mark the region of the defect, or automatic defect detection maybe employed according to a preferred embodiment herein, while theprocess of delineating the defect is also preferably also performedautomatically.

Techniques and means to allow automatic alteration of defects in digitalimages may be based upon a defect channel having a signal proportionalto defects in the digital image (see also U.S. Pat. No. 6,487,321 toEdgar et al., hereby incorporated by reference). This allows areas ofstrong defect to be more easily excised without causing significantdamage to the area of the image surrounding the defect.

Techniques and means may be employed to generate replacement data valuesfor an image region (see also U.S. Pat. No. 6,587,592 to Georgiev etal., hereby incorporated by reference) Image defect may be repaired asfacilitated by the replacement data. Moreover, the repairing of theunwanted image region may preserves image textures within the repaired(or “healed”) region of the image.

Techniques and means may be employed to detect defect pixels by applyinga median filter to an image and subtracting the result from the originalimage to obtain a difference image (see also US patent application2003/0039402 and WIPO patent application WO-03/019473, both to Robins etal., each hereby incorporated by reference). This may be used toconstruct at least one defect map. Correction of suspected defect pixelsmay be achieved by replacing those pixel values with pixel values fromthe filtered image and applying a smoothing operation. User input may ormay not be utilized to further mitigate the effects of uncertainty indefect identification.

Techniques and means for retouching binary image data which is to bepresented on a view-screen or display apparatus may be employed toeliminate local screen defects such as dust and scratch artifacts (seealso US patent application 2002/0154831 to Hansen et al., herebyincorporated by reference). The production of visible moiré effects inthe retouched image data may be avoided by the replacement of smallareas.

A digital video camera with sensor apparatus may incorporate a defectdetecting mode (see also U.S. Pat. No. 5,416,516 to Kameyama et al.,hereby incorporated by reference). The locations of detected defectpixels may be retained in the memory of the camera apparatus andreplacement pixel values may be interpolated by processing algorithms,which convert the sensor data into digital image pixel values.Techniques may be employed to automatically detect and compensate fordefective sensor pixels within a video camera (see also U.S. Pat. No.5,625,413 to Katoh et al., hereby incorporated by reference). The cameramay perform a dark current measurement on start-up when the camera irisis closed and by raising the gain can determine pixels which exhibitabnormally high dark current values. The location of these pixels isrecorded in camera memory as a LUT with associated threshold brightnessvalues associated with each pixel depending on its dark current value;defect compensation depends on input image brightness and ambienttemperature.

An image pickup apparatus, such as a digital camera, may have adetachable lens (see also US patent application 2003/0133027 to Itoh,hereby incorporated by reference). The camera may incorporate a defectdetecting section and a compensation section for sensor defects. Furtherthe defect detection operation may become active when the camera lens isdetached so that the user will not miss an opportunity to take a picturedue to the operation of the defect detection process.

The techniques of the preferred and alternative embodiments describedherein may be applied to printers and to to imaging devices such as adigital cameras which incorporate a focusing lens system. A process maybe employed for detecting and mapping dust on the surface of aphotographic element (see also U.S. Pat. No. 5,436,979 to Gray et al.,hereby incorporated by reference). This may be applied in the context ofa verification procedure to follow a cleaning process for a range ofphotographic elements including film negatives and slides. Statisticalinformation may be obtained and presented to an operator to allowcontrol of the cleaning process. Detailed location information may bealso recorded and/or correction means may be also provided for dustdefects on a photographic element.

Techniques and means to create a defect map for a digital camera orsimilar imaging device may use an all-white reference background (seealso US patent application 2002/0093577 to Kitawaki et al., herebyincorporated by reference). The location of any dust or scratch defectsmay be recorded in the memory of the imaging apparatus when the camerais in a dust detection mode and when a dust correction circuit is activeany image data co-located with a defect may be corrected for thepresence of dust by elimination, color correction or interpolation basedon the surrounding pixels. Further, where the position of a dust defectchanges with f-stop of the camera a list of dust locations correspondingto f-stop settings is pre recorded at the time of manufacturing in a LUTin the camera memory. Any effect of different focal length may besimplified to the effect of the change in dust due to magnification ofthe lens. In addition, techniques for dynamically obtaining a defect mapbased on the processing of a plurality of images may be employed withthis technique.

Techniques may be also employed involving correcting for dust defectsbased on the geometry of said dust or of the camera. Further techniquesmay involve utilizing camera metadata to enhance the detection andcorrection processes for defects. Further techniques may involvealerting the user that the camera requires servicing due to excessivelevels of dust contamination, or the fact that it is not onlymagnification but the actual lens that is mounted.

A method of filtering dust artifacts form an acquired digital imageincluding multiplicity of pixels indicative of dust, the pixels formingvarious shapes in the image, may be employed. The method may includeanalyzing image information including information describing conditionsunder which the image was acquired and/or acquisition device-specificinformation. One or more regions may be determined within the digitalimage suspected as including dust artifact. Based at least in part onsaid meta-data analysis, it may be determined whether the regions areactual dust artifact.

A method may includes obtaining a dust map based on analysis of multipleimages acquired by the same image acquisition device. The dust map mayinclude regions of various shapes and sizes indicative of statisticallyrecurring patterns associated with dust.

A method may further include analyzing the images in comparison to apredetermined dust map to establish the validity of the dust overprogressions of time. The method may further involve mapping theacquired image to a predetermined default acquisition condition as afunction of the lens type and the focal length that was used atacquisition.

A method may further include mapping a dust spec as depicted in the dustmap and the suspected dust specs in the acquired image based on acalculated transformation of the dust as a function of the lens and theaperture, or other extracted parameter, used to acquire the image. Theactual removal of dust artifacts from an image may include a step wheremissing data as obscured by the dust specs is regenerated and in-paintedbased on analysis of the region in the image surrounding the dust spec.The actual removal of the dust artifacts from the image may also includea step where deteriorated regions primarily in the periphery of the dustspec are enhanced and restored based on knowledge of the deteriorationfunction. The actual image retouching may include both in-painting andrestoration or either one of these operations, or another imagecorrection technique as may be understood by those skilled in the art.

A method of detecting and removing dust artifacts may be performed inthe acquisition device as a post-processing stage prior to saving theimage. This method may further include an analysis of the image in itsraw format immediately followed by the acquisition stage. The method ofdetecting and removing dust artifacts can be performed on an externaldevice as part of a download or capture process. Such external devicemay be a personal computer, a storage device, and archival device, adisplay or a printing device or other device. The method of detectingand removing dust artifacts can be performed in part in the acquisitiondevice and the external device.

A dust detection and/or correction technique may be applied post priorito a collection of images, or individually to images as they are addedto a collection. The map may be generated a priori to the introductionof an image, or dynamically and in concurrence to the introduction ofnew images. The method may further include steps of providing astatistical confidence level as to the fact that a certain region isindeed part of a dust spec. The method may further provide tools todetermine whether the acquisition device may benefit from somemaintenance. A method may be employed that may be implemented as part ofa digitization process, such as correcting defects on scanning device,whether flat bed or drum, whether for hard copy documents or for filmdigitization. A method may be further applied to other recurring imageimperfections such as dead pixels on the sensor, burnt pixels on thesensor, scratches, etc. A method of automatically determining whether torecommend servicing a digital image acquisition system including adigital camera based on dust analysis may be advantageously employed. Amethod of calculating parameters of an optical system may be based onanalysis of the dust. While an exemplary drawings and specificembodiments of the present invention have been described andillustrated, it is to be understood that that the scope of the presentinvention is not to be limited to the particular embodiments discussed.Thus, the embodiments shall be regarded as illustrative rather thanrestrictive, and it should be understood that variations may be made inthose embodiments by workers skilled in the arts without departing fromthe scope of the present invention as set forth in the claims thatfollow and their structural and functional equivalents.

In addition, in methods that may be performed according to preferredembodiments herein, the operations have been described in selectedtypographical sequences. However, the sequences have been selected andso ordered for typographical convenience and are not intended to implyany particular order for performing the operations, unless a particularordering is expressly provided or understood by those skilled in the artas being necessary.

Many references have been cited above herein, and in addition to thatwhich is described as background, the invention summary, briefdescription of the drawings, the drawings and the abstract, thesereferences are hereby incorporated by reference into the detaileddescription of the preferred embodiments, as disclosing alternativeembodiments of elements or features of the preferred embodiments nototherwise set forth in detail above. A single one or a combination oftwo or more of these references may be consulted to obtain a variationof the preferred embodiments described in the detailed descriptionabove.

1. A digital image acquisition and processing system that automaticallycorrects dust artifact regions within acquired images by compiling astatistical dust map from multiple images acquired under different imageacquisition conditions, comprising: (A) a digital camera, comprising:(i) an optical system for acquiring an image including a lens assemblyand an aperture stop; and (ii) an electronic sensor array disposedapproximately at an image focal plane of the optical system forcollecting image data according to spectral information associated withmultiple pixels that collectively correspond to the image; and (B) anexternal processing device for coupling with the digital camera toreceive the image data or converted image data internally processedwithin the digital camera, the external device comprising digitalprocessing electronics including a processor for processing the imagedata according to programming instructions; and (C) wherein the systemfurther comprises one or more memories having programming instructionsstored therein for performing a method of automatic image correction ofdust defect regions, the image correction method comprising: (i)acquiring multiple original digital images with said digital acquisitiondevice; (ii) determining probabilities that certain pixels correspond todust artifact regions within said images based at least in part on acomparison of suspected dust artifact regions within two or more of saidimages; (iii) associating probable dust artifact regions with one ormore values of one or more extracted parameters relating to the lensassembly of the digital acquisition device when the images wereacquired; (iv) forming a statistical dust map including mapped dustregions based on the dust artifact determining and associating; (v)correcting pixels corresponding to correlated dust artifact regionswithin further digitally-acquired images based on the associatedstatistical dust map.
 2. The system of claim 1, wherein said determiningprobabilities further comprises statistically combining an plurality ofindividual probabilities based on each said regions within two or moresaid images.
 3. The system of claim 1, the method further comprisingeliminating certain suspected dust artifact regions as having aprobability below a first threshold value.
 4. The system of claim 1, themethod further comprising judging certain probable dust artifact regionsas having a probability above a threshold value such as to be subject tosaid correcting operation.
 5. The system of claim 1, wherein saidprobability determining includes weighting suspected dust artifactregions according to one or more predetermined probability weightingassessment conditions.
 6. The system of claim 1, wherein said furtherdigitally-acquired images comprising different images than saidoriginally acquired images.
 7. The system of claim 1, the method furthercomprising correcting pixels corresponding to correlated dust artifactregions within each of said original images based on the associatedstatistical dust map.
 8. The system of claim 1, the method furthercomprising correcting pixels corresponding to correlated dust artifactregions within said original images based on the associated statisticaldust map.
 9. The system of claim 1, the method further comprisingdigitally-acquiring additional images with said digital camera,repeating said determining and associating, and updating saidstatistical dust map including updating said mapped dust regions basedon the additional dust artifact determining and associating.
 10. Thesystem of claim 1, wherein said determining includes dynamicallyupdating said probabilities based on comparisons with suspectedequivalent dust artifact regions within said further digitally-acquiredimages.
 11. The system of claim 1, wherein said determining of saidprobabilities is further based on a pixel analysis of the suspected dustartifact regions in view of predetermined characteristics indicative ofthe presence of a dust artifact region.
 12. The system of claim 1, themethod further comprising validating whether said furtherdigitally-acquired image has non contradicting data that saidprobability that certain pixels correspond to dust artifact regionswithin said further digitally-acquired image prior to correcting pixelscorresponding to correlated dust artifact regions within furtherdigitally-acquired images based on the associated statistical dust map.13. The system of claim 1, said image correction method being performedon a processed image after being converted from raw format to a knownred, green, blue representation.
 14. The system of claim 1, wherein saidcorrecting includes replacing said pixels within said one or moredigitally-acquired images with new pixels.
 15. The system of claim 1,wherein said correcting includes enhancing said values of pixels withinsaid one or more digitally-acquired images.
 16. The system of claim 1,the dust artifact determining including: (I) loading the statisticaldust map; (II) loading extracted parameter information of a presentimage; (III) performing calculations within the statistical dust maphaving extracted parameter variable-dependencies; and (IV) comparingdust artifact detection data with the extracted parameter dependentstatistical dust map data.
 17. The system of claim 1, the dust artifactdetermining including: (I) loading the statistical dust map; (II)loading extracted parameter information of a present image; (III)performing a calculation for relating the statistical dust map with thepresent image according to a selected value of an extracted parameterwhich is otherwise uncorrelated between the present image and the dustmap; and (IV) comparing dust artifact detection data with the nowcorrelated statistical dust map data.
 18. The system of claim 1, whereinsaid suspected dust artifact regions of at least two images compriseshadow regions and partially-occluded aura regions, and the methodfurther comprises a first comparison of said shadow regions and a secondcomparison of said aura regions.
 19. A digital image acquisition andprocessing system that automatically corrects dust artifact regionswithin acquired images by compiling a statistical dust map from multipleimages acquired under different image acquisition conditions,comprising: (A) a digital camera, comprising: (i) an optical system foracquiring an image including a lens assembly and an aperture stop; and(ii) an electronic sensor array disposed approximately at an image focalplane of the optical system for collecting image data according tospectral information associated with multiple pixels that collectivelycorrespond to the image; and (B) an external processing device forcoupling with the digital camera to receive the image data or convertedimage data internally processed within the digital camera, the externaldevice comprising digital processing electronics including a processorfor processing the image data according to programming instructions; and(C) wherein the system further comprises one or more memories havingprogramming instructions stored therein for performing a method ofautomatic image correction of dust defect regions, the image correctionmethod comprising: (i) digitally-acquiring one or more original imageswith said digital camera; (ii) determining probabilities that certainpixels correspond to dust artifact regions within said one or moredigitally-acquired images; (iii) associating the dust artifact regionswith one or more extracted parameters relating to the optical systemwhen the one or more images were acquired; (iv) forming a statisticaldust map including mapped dust regions based on the dust artifactprobability determining and associating; (v) correcting pixelscorresponding to dust artifact regions within each of said one or moreoriginal images based on the associated statistical dust map.
 20. Thesystem of claim 19, the method further comprising digitally-acquiringfurther images with said digital camera, repeating said determining andassociating, and updating said statistical dust map including updatingsaid mapped dust regions based on the further dust artifact determiningand associating.
 21. The system of claim 19, said determining comprisingdetermining probabilities that certain pixels correspond to dustartifact regions within said acquired images based at least in part on acomparison of suspected dust artifact regions within two or moredigitally-acquired images, or on a pixel analysis of the suspected dustartifact regions in view of predetermined characteristics indicative ofthe presence of a dust artifact region, or both.
 22. The system of claim19, wherein said determining is based at least in part on a comparisonof suspected dust artifact regions within two or more digitally-acquiredimages.
 23. The system of claim 19, wherein said determining of saidprobabilities is further based on a pixel analysis of the suspected dustartifact regions in view of predetermined characteristics indicative ofthe presence of a dust artifact region.
 24. The system of claim 19,wherein said suspected dust artifact regions of said at least two imagescomprise shadow regions and aura regions, and wherein said correctingcomprises a first correcting of said aura regions and a secondcorrecting of said shadow regions.
 25. The system of claim 19, the dustmap including dust artifact regions with probabilities above a thresholdprobability and not including regions with lower probabilities.
 26. Thesystem of claim 19, the dust map including dust artifact regions withprobabilities above a first threshold value, not including regions withprobabilities below a second threshold value, and where further regionshaving a probability between said first and second threshold valuesexist within said dust map, then further image information is combinedinto said probability analysis before determining whether said regionare included as dust artifact regions.
 27. The system of claim 19,wherein said correcting includes replacing said pixels within said oneor more digitally-acquired images with new pixels.
 28. The system ofclaim 19, wherein said correcting includes enhancing said values ofpixels within said one or more digitally-acquired images.
 29. The systemof claim 19, the dust artifact determining operation including: (I)loading the statistical dust map; (II) loading extracted parameterinformation of a present image; (III) performing calculations within thestatistical dust map having extracted parameter variable-dependencies;and (IV) comparing dust artifact detection data with the extractedparameter dependent statistical dust map data.
 30. The system of claim19, the dust artifact determining operation including: (I) loading thestatistical dust map; (II) loading extracted parameter information of apresent image; (III) performing a calculation for relating thestatistical dust map with the present image according to a selectedvalue of an extracted parameter which is otherwise uncorrelated betweenthe present image and the dust map; and (IV) comparing dust artifactdetection data with the now correlated statistical dust map data.
 31. Adigital image acquisition and processing system that automaticallycorrects dust artifact regions within acquired images by compiling astatistical dust map from multiple images acquired under different imageacquisition conditions, comprising: (A) a digital camera, comprising:(i) an optical system for acquiring an image including a lens assemblyand an aperture stop; and (ii) an electronic sensor array disposedapproximately at an image focal plane of the optical system forcollecting image data according to spectral information associated withmultiple pixels that collectively correspond to the image; and (B) anexternal processing device for coupling with the digital camera toreceive the image data or converted image data internally processedwithin the digital camera, the external device comprising digitalprocessing electronics including a processor for processing the imagedata according to programming instructions; and (C) wherein the systemfurther comprises one or more memories having programming instructionsstored therein for performing a method of automatic image correction ofdust defect regions, the image correction method comprising: (i)determining that certain pixels correspond to dust artifact regionswithin a first digitally-acquired image; (ii) associating the dustartifact regions with one or more extracted parameters relating to avariable optical system of said digital camera when the first image wasacquired; (iii) forming an associated statistical dust map includingmapped dust regions based on the dust artifact determining andassociating operations and known correspondence between effects on dustartifact data of one or more different values of said one or moreextracted parameters; and (iv) correcting pixels corresponding tocorrelated dust artifact regions within a second image acquired withsaid digital camera according to one or more different values of saidone or more extracted parameters relating to said variable opticalsystem based on said associated statistical dust map.
 32. The system ofclaim 31, wherein said first and second images were acquired ofdifferent objects.
 33. The system of claim 31, further comprisingcorrecting pixels corresponding to correlated dust artifact regionswithin said first image based on the associated statistical dust map.34. The system of claim 31, the method further comprisingdigitally-acquiring additional images with said digital camera,repeating said determining and associating, and updating saidstatistical dust map including updating said mapped dust regions basedon the additional dust artifact determining and associating.
 35. Thesystem of claim 31, said determining comprising determiningprobabilities that certain pixels correspond to dust artifact regionswithin said first acquired image.
 36. The system of claim 31, whereinsaid determining is based at least in part on a comparison of suspecteddust artifact regions within multiple images including said first imageand at least one other digitally-acquired image.
 37. A digital imageacquisition and processing system that automatically corrects dustartifact regions within acquired images by compiling a statistical dustmap from multiple images acquired under different image acquisitionconditions, comprising: (A) a digital camera, comprising: (i) an opticalsystem for acquiring an image including a lens assembly and an aperturestop; and (ii) an electronic sensor array disposed approximately at animage focal plane of the optical system for collecting image dataaccording to spectral information associated with multiple pixels thatcollectively correspond to the image; and (B) an external processingdevice for coupling with the digital camera to receive the image data orconverted image data internally processed within the digital camera, theexternal device comprising digital processing electronics including aprocessor for processing the image data according to programminginstructions; and (C) wherein the system further comprises one or morememories having programming instructions stored therein for performing amethod of automatic image correction of dust defect regions, the imagecorrection method comprising: (i) determining probabilities that certainpixels within multiple digitally-acquired images correspond to one ormore dust artifact regions; (ii) associating probable dust artifactregions with values of one or more extracted parameters relating to alens system of the digital image acquisition device when the images wereacquired; (iii) forming a statistical dust map including mapped dustregions based on the dust artifact determining and associatingoperations; and (iv) correcting digital data corresponding to pixelswithin one or more digitally-acquired images correlated with mapped dustregions of the statistical dust map.
 38. The system of claim 37, themethod further comprising repeating the dust artifact determining andassociating operations for further acquired images, values of the one ormore extracted parameters relating to the optical system differingbetween at least two acquired images.
 39. The system of claim 38, saidat least two acquired images having been acquired of different objects.40. The system of claim 36, said forming said statistical dust map beingbased upon multiple acquired images and including mathematicallycorrelating dust artifact regions of the multiple acquired imagesaccording to their extracted parameter values including said one or moredifferent extracted parameter values of said at least two acquiredimages.
 41. A digital image acquisition and processing system thatautomatically corrects dust artifact regions within acquired images bycompiling a statistical dust map from multiple images acquired underdifferent image acquisition conditions, comprising: (A) a digitalcamera, comprising: (i) an optical system for acquiring an imageincluding a lens assembly and an aperture stop; and (ii) an electronicsensor array disposed approximately at an image focal plane of theoptical system for collecting image data according to spectralinformation associated with multiple pixels that collectively correspondto the image; and (B) an external processing device for coupling withthe digital camera to receive the image data or converted image datainternally processed within the digital camera, the external devicecomprising digital processing electronics including a processor forprocessing the image data according to programming instructions; and (C)wherein the system further comprises one or more memories havingprogramming instructions stored therein for performing a method ofautomatic image correction of dust defect regions, the image correctionmethod comprising: (i) determining probabilities of dust artifactregions corresponding to pixels within a digitally-acquired image basedat least in part on a pixel analysis of the region in view ofpredetermined characteristics indicative of the presence of a dustartifact region; (ii) associating the dust artifact regions with one ormore extracted parameters relating to the optical system when the imagewas acquired; (iii) forming a statistical dust map includingmapped-dust-regions based on the dust artifact determining andassociating operations; and (iv) correcting the digital datacorresponding to the correlated dust artifact regions within theacquired image based on the associated statistical dust map.
 42. Thesystem of claim 41, the method further comprising eliminating certainsuspected dust artifact regions as having a probability below a firstthreshold value.
 43. The system of claim 41, the method furthercomprising judging certain probable dust artifact regions as having aprobability above a threshold value such as to be subject to saidcorrecting operation.
 44. The system of claim 41, wherein saidprobability determining includes weighting suspected dust artifactregions according to one or more predetermined probability weightingassessment conditions.
 45. The system of claim 41, wherein said furtherdigitally-acquired images comprising different images than saidoriginally acquired images.
 46. The system of claim 41, the methodfurther comprising correcting pixels corresponding to correlated dustartifact regions within each of said original images based on theassociated statistical dust map.
 47. The system of claim 41, the methodfurther comprising correcting pixels corresponding to correlated dustartifact regions within said original images based on the associatedstatistical dust map.
 48. The system of claim 41, wherein saiddetermining includes dynamically updating said probabilities based oncomparisons with suspected equivalent dust artifact regions within saidfurther digitally-acquired images.
 49. The system of claim 41, whereinsaid forming includes dynamically updating said statistical dust mapbased on further pixel analyses of suspected dust artifact regions infurther images in view of predetermined characteristics indicative ofthe presence of a dust artifact region.
 50. The system of claim 41,wherein said suspected dust artifact regions of said at least two imagescomprise shadow regions and aura regions, and wherein said comparisoncomprises a first comparison of said shadow regions and a secondcomparison of said aura regions.
 51. The system of claim 41, whereinsaid determining probabilities further comprises statistically combiningan plurality of individual probabilities based on each said regionswithin two or more said images.
 52. The system of claim 41, the methodfurther comprising validating whether said further digitally-acquiredimage has non contradicting data that said probability that certainpixels correspond to dust artifact regions within said furtherdigitally-acquired image prior to correcting pixels corresponding tocorrelated dust artifact regions within further digitally-acquiredimages based on the associated statistical dust map.
 53. The system ofclaim 41, wherein said correcting includes replacing said pixels withinsaid one or more digitally-acquired images with new pixels.
 54. Thesystem of claim 41, wherein said correcting includes enhancing saidvalues of pixels within said one or more digitally-acquired images. 55.The system of claim 41, the dust artifact determining operationincluding: (I) loading the statistical dust map; (II) loading extractedparameter information of a present image; (III) performing calculationswithin the statistical dust map having extracted parametervariable-dependencies; and (IV) comparing dust artifact detection datawith the extracted parameter dependent statistical dust map data. 56.The system of claim 41, the dust artifact determining operationincluding: (I) loading the statistical dust map; (II) loading extractedparameter information of a present image; (III) performing a calculationfor relating the statistical dust map with the present image accordingto a selected value of an extracted parameter which is otherwiseuncorrelated between the present image and the dust map; and (IV)comparing dust artifact detection data with the now correlatedstatistical dust map data.
 57. The system of claim 41, said dustartifact regions including an aura region partially obscured by dust anda shadow region at least substantially obscured by dust inside said auraregion.
 58. A digital image acquisition and processing system thatautomatically corrects dust artifact regions within acquired images bycompiling a statistical dust map from multiple images acquired underdifferent image acquisition conditions, comprising: (A) a digitalcamera, comprising: (i) an optical system for acquiring an imageincluding a lens assembly and an aperture stop; and (ii) an electronicsensor array disposed approximately at an image focal plane of theoptical system for collecting image data according to spectralinformation associated with multiple pixels that collectively correspondto the image; and (B) an external processing device for coupling withthe digital camera to receive the image data or converted image datainternally processed within the digital camera, the external devicecomprising digital processing electronics including a processor forprocessing the image data according to programming instructions; and (C)wherein the system further comprises one or more memories havingprogramming instructions stored therein for performing a method ofautomatic image correction of dust defect regions, the image correctionmethod comprising: (i) analyzing pixels within one ore more acquireddigital images according to probability determinations that such pixelscorrespond to blemish artifacts; (ii) determining whether a thresholddistribution of blemish artifacts is present within one or more of saiddigital images; and (iii) indicating a need for service when at leastsaid threshold distribution is determined to be present.
 59. The systemof claim 58, the method further comprising: (a) determiningprobabilities of dust artifact regions corresponding to said pixelswithin said digitally-acquired image; (b) associating the dust artifactregions with one or more extracted parameters relating to the lensassembly when the image was acquired; (c) forming a statistical recordincluding dust artifact regions based on the dust artifact determiningand associating; and (d) determining said threshold distribution basedon predetermined characteristics of said statistical record.
 60. Adigital image acquisition and processing system that automaticallycorrects dust artifact regions within acquired images by compiling astatistical dust map from multiple images acquired under different imageacquisition conditions, comprising: (A) a digital camera, comprising:(i) an optical system for acquiring an image including a lens assemblyand an aperture stop; and (ii) an electronic sensor array disposedapproximately at an image focal plane of the optical system forcollecting image data according to spectral information associated withmultiple pixels that collectively correspond to the image; and (B) anexternal processing device for coupling with the digital camera toreceive the image data or converted image data internally processedwithin the digital camera, the external device comprising digitalprocessing electronics including a processor for processing the imagedata according to programming instructions; and (C) wherein the systemfurther comprises one or more memories having programming instructionsstored therein for performing a method of automatic image correction ofdust defect regions, the image correction method comprising: (i)acquiring multiple original digital images with said digital acquisitiondevice; (ii) determining probabilities that certain pixels correspond todust artifact regions within said images based at least in part on acomparison of suspected dust artifact regions within two or more of saidimages; (iii) forming a statistical dust record including probabilitiesof dust artifact regions based on the dust artifact determining andassociating; and (iv) determining a need for service when thestatistical dust record indicates that a predetermined threshold dustartifact distribution is present within digital images acquired withsaid digital acquisition device.
 61. The system of claim 60, whereinsaid probability determining includes weighting suspected dust artifactregions according to one or more predetermined probability weightingassessment conditions.
 62. The system of claim 60, the method furthercomprising associating probable dust artifact regions with one or morevalues of one or more extracted parameters relating to the lens assemblyof the digital acquisition device when the images were acquired, saidstatistical dust record being formed from multiple images including atleast two images having different values of one or more extractedparameters that are mathematically correlated based on known effects ofsaid different values on dust artifact regions appearing within saiddigital images.
 63. A digital image acquisition and processing systemthat automatically corrects dust artifact regions within acquired imagesby compiling a statistical dust map from multiple images acquired underdifferent image acquisition conditions, comprising: (A) a digitalcamera, comprising: (i) an optical system for acquiring an imageincluding a lens assembly and an aperture stop; and (ii) an electronicsensor array disposed approximately at an image focal plane of theoptical system for collecting image data according to spectralinformation associated with multiple pixels that collectively correspondto the image; and (B) an external processing device for coupling withthe digital camera to receive the image data or converted image datainternally processed within the digital camera, the external devicecomprising digital processing electronics including a processor forprocessing the image data according to programming instructions; and (C)wherein the system further comprises one or more memories havingprogramming instructions stored therein for performing a method ofautomatic image correction of dust defect regions, the image correctionmethod comprising: (i) determining probabilities that certain pixelscorrespond to dust artifact regions within a digitally-acquired imagebased at least in part on a pixel analysis of the region in view ofpredetermined characteristics indicative of the presence of a dustartifact region; (ii) forming a statistical dust record includingprobabilities of dust artifact regions based on the dust artifactdetermining and associating operations; and (iii) determining a need forservice when the statistical dust record indicates that a predeterminedthreshold dust artifact distribution is present within digital imagesacquired with said digital acquisition device.
 64. The system of claim63, wherein said probability determining includes weighting suspecteddust artifact regions according to one or more predetermined probabilityweighting assessment conditions.
 65. The system of claim 63, the methodfurther comprising associating probable dust artifact regions with oneor more values of one or more extracted parameters relating to the lensassembly of the digital acquisition device when the images wereacquired, said statistical dust record being formed from multiple imagesincluding at least two images having different values of one or moreextracted parameters that are mathematically correlated based on knowneffects of said different values on dust artifact regions appearingwithin said digital images.
 66. A digital image acquisition andprocessing system that automatically corrects dust artifact regionswithin acquired images by compiling a statistical dust map from multipleimages acquired under different image acquisition conditions,comprising: (A) a digital camera, comprising: (i) an optical system foracquiring an image including a lens assembly and an aperture stop; and(ii) an electronic sensor array disposed approximately at an image focalplane of the optical system for collecting image data according tospectral information associated with multiple pixels that collectivelycorrespond to the image; and (B) an external processing device forcoupling with the digital camera to receive the image data or convertedimage data internally processed within the digital camera, the externaldevice comprising digital processing electronics including a processorfor processing the image data according to programming instructions; and(C) wherein the system further comprises one or more memories havingprogramming instructions stored therein for performing a method ofautomatic image correction of dust defect regions, the image correctionmethod comprising: (i) determining probabilities that certain pixelswithin multiple digitally-acquired images correspond to one or more dustartifact regions; (ii) forming a statistical dust map including mappeddust artifact regions based on the dust artifact determining; (iii)separating said mapped dust artifact regions into aura regions partiallyobscured by dust and shadow regions substantially obscured by dustinside said aura regions; and (iv) correcting digital data correspondingto pixels within one or more digitally-acquired images including dustartifact regions separated into aura regions and shadow regionscorrelated with mapped aura and shadow regions of the statistical dustmap.
 67. The system of claim 66, said dust artifact region probabilitydetermining including separating suspected dust artifact regions intoaura regions and shadow regions, and determining separate probabilitieswhether said aura regions and said shadow regions of said suspected dustartifact regions in fact correspond to dust artifact regions within saidimages.
 68. The system of claim 66, the correcting operation includingcalculating said aura region and said shadow region.
 69. The system ofclaim 66, wherein said dust artifact region probability determining isbased at least in part on a comparison of suspected dust artifactregions within two or more images.
 70. The system of claim 66, whereinsaid dust artifact region probability determining is based at least inpart on a comparison of suspected dust artifact regions within imageswith predetermined characteristics indicative of the presence of a dustartifact region.